Projects
openEuler:24.03:SP1:Everything
python3
Sign Up
Log In
Username
Password
We truncated the diff of some files because they were too big. If you want to see the full diff for every file,
click here
.
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
Expand all
Collapse all
Changes of Revision 3
View file
_service:tar_scm:python3.spec
Changed
@@ -3,7 +3,7 @@ URL: https://www.python.org/ Version: 3.11.6 -Release: 6 +Release: 7 License: Python-2.0 %global branchversion 3.11 @@ -88,12 +88,17 @@ Patch1: 00001-rpath.patch Patch251: 00251-change-user-install-location.patch -Patch6000: backport-3.11-gh-114572-Fix-locking-in-cert_store_stats-and-g.patch -Patch6001: backport-3.11-gh-113171-gh-65056-Fix-private-non-global-IP-ad.patch +Patch6000: backport-CVE-2024-0397-gh-114572-Fix-locking-in-cert_store_stats-and-g.patch +Patch6001: backport-CVE-2024-4032-gh-113171-gh-65056-Fix-private-non-global-IP-ad.patch Patch6002: backport-fix_xml_tree_assert_error.patch -Patch6003: backport-gh-121650-Encode-newlines-in-headers-and-verify-head.patch -Patch6004: backport-gh-123067-Fix-quadratic-complexity-in-parsing-quoted.patch -Patch6005: backport-gh-123270-Replaced-SanitizedNames-with-a-more-surgic.patch +Patch6003: backport-CVE-2024-6923-gh-121650-Encode-newlines-in-headers-and-verify-head.patch +Patch6004: backport-CVE-2024-7592-gh-123067-Fix-quadratic-complexity-in-parsing-quoted.patch +Patch6005: backport-CVE-2024-8088-gh-123270-Replaced-SanitizedNames-with-a-more-surgic.patch +Patch6006: backport-CVE-2024-6232-gh-121285-Remove-backtracking-when-parsing-tarf.patch +Patch6007: backport-CVE-2024-3219-1-gh-122133-Authenticate-socket-connection-for-so.patch +Patch6008: backport-CVE-2024-3219-2-gh-122133-Rework-pure-Python-socketpair-tests-t.patch +Patch6009: backport-CVE-2023-6597-gh-91133-tempfile.TemporaryDirectory-fix-symlin.patch +Patch6010: backport-CVE-2024-0450-gh-109858-Protect-zipfile-from-quoted-overlap-z.patch Patch9000: add-the-sm3-method-for-obtaining-the-salt-value.patch Patch9001: 0001-add-loongarch64-support-for-python.patch @@ -198,7 +203,11 @@ %patch6003 -p1 %patch6004 -p1 %patch6005 -p1 - +%patch6006 -p1 +%patch6007 -p1 +%patch6008 -p1 +%patch6009 -p1 +%patch6010 -p1 %patch9000 -p1 %patch9001 -p1 @@ -863,6 +872,18 @@ %{_mandir}/*/* %changelog +* Tue Sep 24 2024 xinsheng <xinsheng3@huawei.com> - 3.11.6-7 +- Type:CVE +- CVE:CVE-2024-6232,CVE-2024-3219,CVE-2024-0450,CVE-2023-6597 +- SUG:NA +- DESC:fix CVE-2024-6232,CVE-2024-3219,CVE-2024-0450,CVE-2023-6597 + - rename all CVE patch name + - CVE-2024-6232: Remove backtracking when parsing tarfile headers + - CVE-2024-3219: patch1 Authenticate socket connection for `socket.socketpair()` fallback + - CVE-2024-3219: patch2 Rework pure Python socketpair tests to avoid use of importlib.reload. + - CVE-2024-0450: Protect zipfile from "quoted-overlap" zipbomb + - CVE-2023-6597: tempfile.TemporaryDirectory: fix symlink bug in cleanup + * Tue Sep 03 2024 xinsheng <xinsheng3@huawei.com> - 3.11.6-6 - Type:CVE - CVE:NA
View file
_service:tar_scm:backport-3.11-gh-113171-gh-65056-Fix-private-non-global-IP-ad.patch
Deleted
@@ -1,346 +0,0 @@ -From ba431579efdcbaed7a96f2ac4ea0775879a332fb Mon Sep 17 00:00:00 2001 -From: Petr Viktorin <encukou@gmail.com> -Date: Thu, 25 Apr 2024 14:45:48 +0200 -Subject: PATCH 3.11 gh-113171: gh-65056: Fix "private" (non-global) IP - address ranges (GH-113179) (GH-113186) (GH-118177) (#118227) - ---- - Doc/library/ipaddress.rst | 43 +++++++- - Doc/whatsnew/3.11.rst | 9 ++ - Lib/ipaddress.py | 99 +++++++++++++++---- - Lib/test/test_ipaddress.py | 21 +++- - ...-03-14-01-38-44.gh-issue-113171.VFnObz.rst | 9 ++ - 5 files changed, 157 insertions(+), 24 deletions(-) - create mode 100644 Misc/NEWS.d/next/Library/2024-03-14-01-38-44.gh-issue-113171.VFnObz.rst - -diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst -index 03dc956cd1..f57fa15aa5 100644 ---- a/Doc/library/ipaddress.rst -+++ b/Doc/library/ipaddress.rst -@@ -178,18 +178,53 @@ write code that handles both IP versions correctly. Address objects are - - .. attribute:: is_private - -- ``True`` if the address is allocated for private networks. See -+ ``True`` if the address is defined as not globally reachable by - iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ -- (for IPv6). -+ (for IPv6) with the following exceptions: -+ -+ * ``is_private`` is ``False`` for the shared address space (``100.64.0.0/10``) -+ * For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the -+ semantics of the underlying IPv4 addresses and the following condition holds -+ (see :attr:`IPv6Address.ipv4_mapped`):: -+ -+ address.is_private == address.ipv4_mapped.is_private -+ -+ ``is_private`` has value opposite to :attr:`is_global`, except for the shared address space -+ (``100.64.0.0/10`` range) where they are both ``False``. -+ -+ .. versionchanged:: 3.11.10 -+ -+ Fixed some false positives and false negatives. -+ -+ * ``192.0.0.0/24`` is considered private with the exception of ``192.0.0.9/32`` and -+ ``192.0.0.10/32`` (previously: only the ``192.0.0.0/29`` sub-range was considered private). -+ * ``64:ff9b:1::/48`` is considered private. -+ * ``2002::/16`` is considered private. -+ * There are exceptions within ``2001::/23`` (otherwise considered private): ``2001:1::1/128``, -+ ``2001:1::2/128``, ``2001:3::/32``, ``2001:4:112::/48``, ``2001:20::/28``, ``2001:30::/28``. -+ The exceptions are not considered private. - - .. attribute:: is_global - -- ``True`` if the address is allocated for public networks. See -+ ``True`` if the address is defined as globally reachable by - iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ -- (for IPv6). -+ (for IPv6) with the following exception: -+ -+ For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the -+ semantics of the underlying IPv4 addresses and the following condition holds -+ (see :attr:`IPv6Address.ipv4_mapped`):: -+ -+ address.is_global == address.ipv4_mapped.is_global -+ -+ ``is_global`` has value opposite to :attr:`is_private`, except for the shared address space -+ (``100.64.0.0/10`` range) where they are both ``False``. - - .. versionadded:: 3.4 - -+ .. versionchanged:: 3.11.10 -+ -+ Fixed some false positives and false negatives, see :attr:`is_private` for details. -+ - .. attribute:: is_unspecified - - ``True`` if the address is unspecified. See :RFC:`5735` (for IPv4) -diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst -index f670fa1f09..42b61c75c7 100644 ---- a/Doc/whatsnew/3.11.rst -+++ b/Doc/whatsnew/3.11.rst -@@ -2727,3 +2727,12 @@ OpenSSL - * Windows builds and macOS installers from python.org now use OpenSSL 3.0. - - .. _libb2: https://www.blake2.net/ -+ -+Notable changes in 3.11.10 -+========================== -+ -+ipaddress -+--------- -+ -+* Fixed ``is_global`` and ``is_private`` behavior in ``IPv4Address``, -+ ``IPv6Address``, ``IPv4Network`` and ``IPv6Network``. -diff --git a/Lib/ipaddress.py b/Lib/ipaddress.py -index 16ba16cd7d..567beb37e0 100644 ---- a/Lib/ipaddress.py -+++ b/Lib/ipaddress.py -@@ -1086,7 +1086,11 @@ def is_private(self): - """ - return any(self.network_address in priv_network and - self.broadcast_address in priv_network -- for priv_network in self._constants._private_networks) -+ for priv_network in self._constants._private_networks) and all( -+ self.network_address not in network and -+ self.broadcast_address not in network -+ for network in self._constants._private_networks_exceptions -+ ) - - @property - def is_global(self): -@@ -1333,18 +1337,41 @@ def is_reserved(self): - @property - @functools.lru_cache() - def is_private(self): -- """Test if this address is allocated for private networks. -+ """``True`` if the address is defined as not globally reachable by -+ iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ -+ (for IPv6) with the following exceptions: - -- Returns: -- A boolean, True if the address is reserved per -- iana-ipv4-special-registry. -+ * ``is_private`` is ``False`` for ``100.64.0.0/10`` -+ * For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the -+ semantics of the underlying IPv4 addresses and the following condition holds -+ (see :attr:`IPv6Address.ipv4_mapped`):: - -+ address.is_private == address.ipv4_mapped.is_private -+ -+ ``is_private`` has value opposite to :attr:`is_global`, except for the ``100.64.0.0/10`` -+ IPv4 range where they are both ``False``. - """ -- return any(self in net for net in self._constants._private_networks) -+ return ( -+ any(self in net for net in self._constants._private_networks) -+ and all(self not in net for net in self._constants._private_networks_exceptions) -+ ) - - @property - @functools.lru_cache() - def is_global(self): -+ """``True`` if the address is defined as globally reachable by -+ iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ -+ (for IPv6) with the following exception: -+ -+ For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the -+ semantics of the underlying IPv4 addresses and the following condition holds -+ (see :attr:`IPv6Address.ipv4_mapped`):: -+ -+ address.is_global == address.ipv4_mapped.is_global -+ -+ ``is_global`` has value opposite to :attr:`is_private`, except for the ``100.64.0.0/10`` -+ IPv4 range where they are both ``False``. -+ """ - return self not in self._constants._public_network and not self.is_private - - @property -@@ -1548,13 +1575,15 @@ class _IPv4Constants: - - _public_network = IPv4Network('100.64.0.0/10') - -+ # Not globally reachable address blocks listed on -+ # https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml - _private_networks = - IPv4Network('0.0.0.0/8'), - IPv4Network('10.0.0.0/8'), - IPv4Network('127.0.0.0/8'), - IPv4Network('169.254.0.0/16'), - IPv4Network('172.16.0.0/12'), -- IPv4Network('192.0.0.0/29'), -+ IPv4Network('192.0.0.0/24'), - IPv4Network('192.0.0.170/31'), - IPv4Network('192.0.2.0/24'), - IPv4Network('192.168.0.0/16'), -@@ -1565,6 +1594,11 @@ class _IPv4Constants: - IPv4Network('255.255.255.255/32'), - - -+ _private_networks_exceptions = -+ IPv4Network('192.0.0.9/32'), -+ IPv4Network('192.0.0.10/32'), -+ -+ - _reserved_network = IPv4Network('240.0.0.0/4') - - _unspecified_address = IPv4Address('0.0.0.0') -@@ -2010,27 +2044,42 @@ def is_site_local(self): - @property - @functools.lru_cache() - def is_private(self): -- """Test if this address is allocated for private networks. -+ """``True`` if the address is defined as not globally reachable by -+ iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ -+ (for IPv6) with the following exceptions: - -- Returns: -- A boolean, True if the address is reserved per -- iana-ipv6-special-registry, or is ipv4_mapped and is
View file
_service:tar_scm:backport-3.11-gh-114572-Fix-locking-in-cert_store_stats-and-g.patch
Deleted
@@ -1,156 +0,0 @@ -From 01c37f1d0714f5822d34063ca7180b595abf589d Mon Sep 17 00:00:00 2001 -From: "Miss Islington (bot)" - <31488909+miss-islington@users.noreply.github.com> -Date: Tue, 20 Feb 2024 17:34:44 +0100 -Subject: PATCH 3.11 gh-114572: Fix locking in cert_store_stats and - get_ca_certs (GH-114573) (#115549) - -gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573) - -* gh-114572: Fix locking in cert_store_stats and get_ca_certs - -cert_store_stats and get_ca_certs query the SSLContext's X509_STORE with -X509_STORE_get0_objects, but reading the result requires a lock. See -https://github.com/openssl/openssl/pull/23224 for details. - -Instead, use X509_STORE_get1_objects, newly added in that PR. -X509_STORE_get1_objects does not exist in current OpenSSLs, but we can -polyfill it with X509_STORE_lock and X509_STORE_unlock. - -* Work around const-correctness problem - -* Add missing X509_STORE_get1_objects failure check - -* Add blurb -(cherry picked from commit bce693111bff906ccf9281c22371331aaff766ab) - -Co-authored-by: David Benjamin <davidben@google.com> ---- - ...-01-26-22-14-09.gh-issue-114572.t1QMQD.rst | 4 ++ - Modules/_ssl.c | 65 +++++++++++++++++-- - 2 files changed, 64 insertions(+), 5 deletions(-) - create mode 100644 Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst - -diff --git a/Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst b/Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst -new file mode 100644 -index 0000000000..b4f9fe64db ---- /dev/null -+++ b/Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst -@@ -0,0 +1,4 @@ -+:meth:`ssl.SSLContext.cert_store_stats` and -+:meth:`ssl.SSLContext.get_ca_certs` now correctly lock access to the -+certificate store, when the :class:`ssl.SSLContext` is shared across -+multiple threads. -diff --git a/Modules/_ssl.c b/Modules/_ssl.c -index 67ce6e97af..81d36a6f11 100644 ---- a/Modules/_ssl.c -+++ b/Modules/_ssl.c -@@ -4529,6 +4529,50 @@ set_sni_callback(PySSLContext *self, PyObject *arg, void *c) - return 0; - } - -+#if OPENSSL_VERSION_NUMBER < 0x30300000L -+static X509_OBJECT *x509_object_dup(const X509_OBJECT *obj) -+{ -+ int ok; -+ X509_OBJECT *ret = X509_OBJECT_new(); -+ if (ret == NULL) { -+ return NULL; -+ } -+ switch (X509_OBJECT_get_type(obj)) { -+ case X509_LU_X509: -+ ok = X509_OBJECT_set1_X509(ret, X509_OBJECT_get0_X509(obj)); -+ break; -+ case X509_LU_CRL: -+ /* X509_OBJECT_get0_X509_CRL was not const-correct prior to 3.0.*/ -+ ok = X509_OBJECT_set1_X509_CRL( -+ ret, X509_OBJECT_get0_X509_CRL((X509_OBJECT *)obj)); -+ break; -+ default: -+ /* We cannot duplicate unrecognized types in a polyfill, but it is -+ * safe to leave an empty object. The caller will ignore it. */ -+ ok = 1; -+ break; -+ } -+ if (!ok) { -+ X509_OBJECT_free(ret); -+ return NULL; -+ } -+ return ret; -+} -+ -+static STACK_OF(X509_OBJECT) * -+X509_STORE_get1_objects(X509_STORE *store) -+{ -+ STACK_OF(X509_OBJECT) *ret; -+ if (!X509_STORE_lock(store)) { -+ return NULL; -+ } -+ ret = sk_X509_OBJECT_deep_copy(X509_STORE_get0_objects(store), -+ x509_object_dup, X509_OBJECT_free); -+ X509_STORE_unlock(store); -+ return ret; -+} -+#endif -+ - PyDoc_STRVAR(PySSLContext_sni_callback_doc, - "Set a callback that will be called when a server name is provided by the SSL/TLS client in the SNI extension.\n\ - \n\ -@@ -4558,7 +4602,12 @@ _ssl__SSLContext_cert_store_stats_impl(PySSLContext *self) - int x509 = 0, crl = 0, ca = 0, i; - - store = SSL_CTX_get_cert_store(self->ctx); -- objs = X509_STORE_get0_objects(store); -+ objs = X509_STORE_get1_objects(store); -+ if (objs == NULL) { -+ PyErr_SetString(PyExc_MemoryError, "failed to query cert store"); -+ return NULL; -+ } -+ - for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { - obj = sk_X509_OBJECT_value(objs, i); - switch (X509_OBJECT_get_type(obj)) { -@@ -4572,12 +4621,11 @@ _ssl__SSLContext_cert_store_stats_impl(PySSLContext *self) - crl++; - break; - default: -- /* Ignore X509_LU_FAIL, X509_LU_RETRY, X509_LU_PKEY. -- * As far as I can tell they are internal states and never -- * stored in a cert store */ -+ /* Ignore unrecognized types. */ - break; - } - } -+ sk_X509_OBJECT_pop_free(objs, X509_OBJECT_free); - return Py_BuildValue("{sisisi}", "x509", x509, "crl", crl, - "x509_ca", ca); - } -@@ -4609,7 +4657,12 @@ _ssl__SSLContext_get_ca_certs_impl(PySSLContext *self, int binary_form) - } - - store = SSL_CTX_get_cert_store(self->ctx); -- objs = X509_STORE_get0_objects(store); -+ objs = X509_STORE_get1_objects(store); -+ if (objs == NULL) { -+ PyErr_SetString(PyExc_MemoryError, "failed to query cert store"); -+ goto error; -+ } -+ - for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { - X509_OBJECT *obj; - X509 *cert; -@@ -4637,9 +4690,11 @@ _ssl__SSLContext_get_ca_certs_impl(PySSLContext *self, int binary_form) - } - Py_CLEAR(ci); - } -+ sk_X509_OBJECT_pop_free(objs, X509_OBJECT_free); - return rlist; - - error: -+ sk_X509_OBJECT_pop_free(objs, X509_OBJECT_free); - Py_XDECREF(ci); - Py_XDECREF(rlist); - return NULL; --- -2.27.0 -
View file
_service:tar_scm:backport-CVE-2023-6597-gh-91133-tempfile.TemporaryDirectory-fix-symlin.patch
Added
@@ -0,0 +1,218 @@ +From 5585334d772b253a01a6730e8202ffb1607c3d25 Mon Sep 17 00:00:00 2001 +From: Serhiy Storchaka <storchaka@gmail.com> +Date: Thu, 7 Dec 2023 18:37:10 +0200 +Subject: PATCH 3.11 gh-91133: tempfile.TemporaryDirectory: fix symlink bug + in cleanup (GH-99930) (GH-112839) +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +(cherry picked from commit 81c16cd94ec38d61aa478b9a452436dc3b1b524d) + +Co-authored-by: Søren Løvborg <sorenl@unity3d.com> +--- + Lib/tempfile.py | 27 +++-- + Lib/test/test_tempfile.py | 111 +++++++++++++++++- + ...2-12-01-16-57-44.gh-issue-91133.LKMVCV.rst | 2 + + 3 files changed, 125 insertions(+), 15 deletions(-) + create mode 100644 Misc/NEWS.d/next/Library/2022-12-01-16-57-44.gh-issue-91133.LKMVCV.rst + +diff --git a/Lib/tempfile.py b/Lib/tempfile.py +index aace11fa7b1..f59a63a7b45 100644 +--- a/Lib/tempfile.py ++++ b/Lib/tempfile.py +@@ -270,6 +270,22 @@ def _mkstemp_inner(dir, pre, suf, flags, output_type): + raise FileExistsError(_errno.EEXIST, + "No usable temporary file name found") + ++def _dont_follow_symlinks(func, path, *args): ++ # Pass follow_symlinks=False, unless not supported on this platform. ++ if func in _os.supports_follow_symlinks: ++ func(path, *args, follow_symlinks=False) ++ elif _os.name == 'nt' or not _os.path.islink(path): ++ func(path, *args) ++ ++def _resetperms(path): ++ try: ++ chflags = _os.chflags ++ except AttributeError: ++ pass ++ else: ++ _dont_follow_symlinks(chflags, path, 0) ++ _dont_follow_symlinks(_os.chmod, path, 0o700) ++ + + # User visible interfaces. + +@@ -863,17 +879,10 @@ def __init__(self, suffix=None, prefix=None, dir=None, + def _rmtree(cls, name, ignore_errors=False): + def onerror(func, path, exc_info): + if issubclass(exc_info0, PermissionError): +- def resetperms(path): +- try: +- _os.chflags(path, 0) +- except AttributeError: +- pass +- _os.chmod(path, 0o700) +- + try: + if path != name: +- resetperms(_os.path.dirname(path)) +- resetperms(path) ++ _resetperms(_os.path.dirname(path)) ++ _resetperms(path) + + try: + _os.unlink(path) +diff --git a/Lib/test/test_tempfile.py b/Lib/test/test_tempfile.py +index 1242ec7e3cc..675edc8de9c 100644 +--- a/Lib/test/test_tempfile.py ++++ b/Lib/test/test_tempfile.py +@@ -1565,6 +1565,103 @@ def test_cleanup_with_symlink_to_a_directory(self): + "were deleted") + d2.cleanup() + ++ @os_helper.skip_unless_symlink ++ def test_cleanup_with_symlink_modes(self): ++ # cleanup() should not follow symlinks when fixing mode bits (#91133) ++ with self.do_create(recurse=0) as d2: ++ file1 = os.path.join(d2, 'file1') ++ open(file1, 'wb').close() ++ dir1 = os.path.join(d2, 'dir1') ++ os.mkdir(dir1) ++ for mode in range(8): ++ mode <<= 6 ++ with self.subTest(mode=format(mode, '03o')): ++ def test(target, target_is_directory): ++ d1 = self.do_create(recurse=0) ++ symlink = os.path.join(d1.name, 'symlink') ++ os.symlink(target, symlink, ++ target_is_directory=target_is_directory) ++ try: ++ os.chmod(symlink, mode, follow_symlinks=False) ++ except NotImplementedError: ++ pass ++ try: ++ os.chmod(symlink, mode) ++ except FileNotFoundError: ++ pass ++ os.chmod(d1.name, mode) ++ d1.cleanup() ++ self.assertFalse(os.path.exists(d1.name)) ++ ++ with self.subTest('nonexisting file'): ++ test('nonexisting', target_is_directory=False) ++ with self.subTest('nonexisting dir'): ++ test('nonexisting', target_is_directory=True) ++ ++ with self.subTest('existing file'): ++ os.chmod(file1, mode) ++ old_mode = os.stat(file1).st_mode ++ test(file1, target_is_directory=False) ++ new_mode = os.stat(file1).st_mode ++ self.assertEqual(new_mode, old_mode, ++ '%03o != %03o' % (new_mode, old_mode)) ++ ++ with self.subTest('existing dir'): ++ os.chmod(dir1, mode) ++ old_mode = os.stat(dir1).st_mode ++ test(dir1, target_is_directory=True) ++ new_mode = os.stat(dir1).st_mode ++ self.assertEqual(new_mode, old_mode, ++ '%03o != %03o' % (new_mode, old_mode)) ++ ++ @unittest.skipUnless(hasattr(os, 'chflags'), 'requires os.chflags') ++ @os_helper.skip_unless_symlink ++ def test_cleanup_with_symlink_flags(self): ++ # cleanup() should not follow symlinks when fixing flags (#91133) ++ flags = stat.UF_IMMUTABLE | stat.UF_NOUNLINK ++ self.check_flags(flags) ++ ++ with self.do_create(recurse=0) as d2: ++ file1 = os.path.join(d2, 'file1') ++ open(file1, 'wb').close() ++ dir1 = os.path.join(d2, 'dir1') ++ os.mkdir(dir1) ++ def test(target, target_is_directory): ++ d1 = self.do_create(recurse=0) ++ symlink = os.path.join(d1.name, 'symlink') ++ os.symlink(target, symlink, ++ target_is_directory=target_is_directory) ++ try: ++ os.chflags(symlink, flags, follow_symlinks=False) ++ except NotImplementedError: ++ pass ++ try: ++ os.chflags(symlink, flags) ++ except FileNotFoundError: ++ pass ++ os.chflags(d1.name, flags) ++ d1.cleanup() ++ self.assertFalse(os.path.exists(d1.name)) ++ ++ with self.subTest('nonexisting file'): ++ test('nonexisting', target_is_directory=False) ++ with self.subTest('nonexisting dir'): ++ test('nonexisting', target_is_directory=True) ++ ++ with self.subTest('existing file'): ++ os.chflags(file1, flags) ++ old_flags = os.stat(file1).st_flags ++ test(file1, target_is_directory=False) ++ new_flags = os.stat(file1).st_flags ++ self.assertEqual(new_flags, old_flags) ++ ++ with self.subTest('existing dir'): ++ os.chflags(dir1, flags) ++ old_flags = os.stat(dir1).st_flags ++ test(dir1, target_is_directory=True) ++ new_flags = os.stat(dir1).st_flags ++ self.assertEqual(new_flags, old_flags) ++ + @support.cpython_only + def test_del_on_collection(self): + # A TemporaryDirectory is deleted when garbage collected +@@ -1737,10 +1834,7 @@ def test_modes(self): + d.cleanup() + self.assertFalse(os.path.exists(d.name)) + +- @unittest.skipUnless(hasattr(os, 'chflags'), 'requires os.chflags') +- def test_flags(self): +- flags = stat.UF_IMMUTABLE | stat.UF_NOUNLINK +- ++ def check_flags(self, flags): + # skip the test if these flags are not supported (ex: FreeBSD 13) + filename = os_helper.TESTFN + try: +@@ -1749,13 +1843,18 @@ def test_flags(self): + os.chflags(filename, flags) + except OSError as exc: + # "OSError: Errno 45 Operation not supported" +- self.skipTest(f"chflags() doesn't support " +- f"UF_IMMUTABLE|UF_NOUNLINK: {exc}") ++ self.skipTest(f"chflags() doesn't support flags " ++ f"{flags:#b}: {exc}") + else: + os.chflags(filename, 0) + finally: + os_helper.unlink(filename) +
View file
_service:tar_scm:backport-CVE-2024-0397-gh-114572-Fix-locking-in-cert_store_stats-and-g.patch
Added
@@ -0,0 +1,156 @@ +From 01c37f1d0714f5822d34063ca7180b595abf589d Mon Sep 17 00:00:00 2001 +From: "Miss Islington (bot)" + <31488909+miss-islington@users.noreply.github.com> +Date: Tue, 20 Feb 2024 17:34:44 +0100 +Subject: PATCH 3.11 gh-114572: Fix locking in cert_store_stats and + get_ca_certs (GH-114573) (#115549) + +gh-114572: Fix locking in cert_store_stats and get_ca_certs (GH-114573) + +* gh-114572: Fix locking in cert_store_stats and get_ca_certs + +cert_store_stats and get_ca_certs query the SSLContext's X509_STORE with +X509_STORE_get0_objects, but reading the result requires a lock. See +https://github.com/openssl/openssl/pull/23224 for details. + +Instead, use X509_STORE_get1_objects, newly added in that PR. +X509_STORE_get1_objects does not exist in current OpenSSLs, but we can +polyfill it with X509_STORE_lock and X509_STORE_unlock. + +* Work around const-correctness problem + +* Add missing X509_STORE_get1_objects failure check + +* Add blurb +(cherry picked from commit bce693111bff906ccf9281c22371331aaff766ab) + +Co-authored-by: David Benjamin <davidben@google.com> +--- + ...-01-26-22-14-09.gh-issue-114572.t1QMQD.rst | 4 ++ + Modules/_ssl.c | 65 +++++++++++++++++-- + 2 files changed, 64 insertions(+), 5 deletions(-) + create mode 100644 Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst + +diff --git a/Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst b/Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst +new file mode 100644 +index 0000000000..b4f9fe64db +--- /dev/null ++++ b/Misc/NEWS.d/next/Security/2024-01-26-22-14-09.gh-issue-114572.t1QMQD.rst +@@ -0,0 +1,4 @@ ++:meth:`ssl.SSLContext.cert_store_stats` and ++:meth:`ssl.SSLContext.get_ca_certs` now correctly lock access to the ++certificate store, when the :class:`ssl.SSLContext` is shared across ++multiple threads. +diff --git a/Modules/_ssl.c b/Modules/_ssl.c +index 67ce6e97af..81d36a6f11 100644 +--- a/Modules/_ssl.c ++++ b/Modules/_ssl.c +@@ -4529,6 +4529,50 @@ set_sni_callback(PySSLContext *self, PyObject *arg, void *c) + return 0; + } + ++#if OPENSSL_VERSION_NUMBER < 0x30300000L ++static X509_OBJECT *x509_object_dup(const X509_OBJECT *obj) ++{ ++ int ok; ++ X509_OBJECT *ret = X509_OBJECT_new(); ++ if (ret == NULL) { ++ return NULL; ++ } ++ switch (X509_OBJECT_get_type(obj)) { ++ case X509_LU_X509: ++ ok = X509_OBJECT_set1_X509(ret, X509_OBJECT_get0_X509(obj)); ++ break; ++ case X509_LU_CRL: ++ /* X509_OBJECT_get0_X509_CRL was not const-correct prior to 3.0.*/ ++ ok = X509_OBJECT_set1_X509_CRL( ++ ret, X509_OBJECT_get0_X509_CRL((X509_OBJECT *)obj)); ++ break; ++ default: ++ /* We cannot duplicate unrecognized types in a polyfill, but it is ++ * safe to leave an empty object. The caller will ignore it. */ ++ ok = 1; ++ break; ++ } ++ if (!ok) { ++ X509_OBJECT_free(ret); ++ return NULL; ++ } ++ return ret; ++} ++ ++static STACK_OF(X509_OBJECT) * ++X509_STORE_get1_objects(X509_STORE *store) ++{ ++ STACK_OF(X509_OBJECT) *ret; ++ if (!X509_STORE_lock(store)) { ++ return NULL; ++ } ++ ret = sk_X509_OBJECT_deep_copy(X509_STORE_get0_objects(store), ++ x509_object_dup, X509_OBJECT_free); ++ X509_STORE_unlock(store); ++ return ret; ++} ++#endif ++ + PyDoc_STRVAR(PySSLContext_sni_callback_doc, + "Set a callback that will be called when a server name is provided by the SSL/TLS client in the SNI extension.\n\ + \n\ +@@ -4558,7 +4602,12 @@ _ssl__SSLContext_cert_store_stats_impl(PySSLContext *self) + int x509 = 0, crl = 0, ca = 0, i; + + store = SSL_CTX_get_cert_store(self->ctx); +- objs = X509_STORE_get0_objects(store); ++ objs = X509_STORE_get1_objects(store); ++ if (objs == NULL) { ++ PyErr_SetString(PyExc_MemoryError, "failed to query cert store"); ++ return NULL; ++ } ++ + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { + obj = sk_X509_OBJECT_value(objs, i); + switch (X509_OBJECT_get_type(obj)) { +@@ -4572,12 +4621,11 @@ _ssl__SSLContext_cert_store_stats_impl(PySSLContext *self) + crl++; + break; + default: +- /* Ignore X509_LU_FAIL, X509_LU_RETRY, X509_LU_PKEY. +- * As far as I can tell they are internal states and never +- * stored in a cert store */ ++ /* Ignore unrecognized types. */ + break; + } + } ++ sk_X509_OBJECT_pop_free(objs, X509_OBJECT_free); + return Py_BuildValue("{sisisi}", "x509", x509, "crl", crl, + "x509_ca", ca); + } +@@ -4609,7 +4657,12 @@ _ssl__SSLContext_get_ca_certs_impl(PySSLContext *self, int binary_form) + } + + store = SSL_CTX_get_cert_store(self->ctx); +- objs = X509_STORE_get0_objects(store); ++ objs = X509_STORE_get1_objects(store); ++ if (objs == NULL) { ++ PyErr_SetString(PyExc_MemoryError, "failed to query cert store"); ++ goto error; ++ } ++ + for (i = 0; i < sk_X509_OBJECT_num(objs); i++) { + X509_OBJECT *obj; + X509 *cert; +@@ -4637,9 +4690,11 @@ _ssl__SSLContext_get_ca_certs_impl(PySSLContext *self, int binary_form) + } + Py_CLEAR(ci); + } ++ sk_X509_OBJECT_pop_free(objs, X509_OBJECT_free); + return rlist; + + error: ++ sk_X509_OBJECT_pop_free(objs, X509_OBJECT_free); + Py_XDECREF(ci); + Py_XDECREF(rlist); + return NULL; +-- +2.27.0 +
View file
_service:tar_scm:backport-CVE-2024-0450-gh-109858-Protect-zipfile-from-quoted-overlap-z.patch
Added
@@ -0,0 +1,146 @@ +From a956e510f6336d5ae111ba429a61c3ade30a7549 Mon Sep 17 00:00:00 2001 +From: "Miss Islington (bot)" + <31488909+miss-islington@users.noreply.github.com> +Date: Thu, 11 Jan 2024 10:24:47 +0100 +Subject: PATCH 3.11 gh-109858: Protect zipfile from "quoted-overlap" + zipbomb (GH-110016) (GH-113913) + +Raise BadZipFile when try to read an entry that overlaps with other entry or +central directory. +(cherry picked from commit 66363b9a7b9fe7c99eba3a185b74c5fdbf842eba) + +Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> +--- + Lib/test/test_zipfile.py | 60 +++++++++++++++++++ + Lib/zipfile.py | 12 ++++ + ...-09-28-13-15-51.gh-issue-109858.43e2dg.rst | 3 + + 3 files changed, 75 insertions(+) + create mode 100644 Misc/NEWS.d/next/Library/2023-09-28-13-15-51.gh-issue-109858.43e2dg.rst + +diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py +index c8e0159765e..9354ab74faa 100644 +--- a/Lib/test/test_zipfile.py ++++ b/Lib/test/test_zipfile.py +@@ -2216,6 +2216,66 @@ def test_decompress_without_3rd_party_library(self): + with zipfile.ZipFile(zip_file) as zf: + self.assertRaises(RuntimeError, zf.extract, 'a.txt') + ++ @requires_zlib() ++ def test_full_overlap(self): ++ data = ( ++ b'PK\x03\x04\x14\x00\x00\x00\x08\x00\xa0lH\x05\xe2\x1e' ++ b'8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00\x00\x00a\xed' ++ b'\xc0\x81\x08\x00\x00\x00\xc00\xd6\xfbK\\d\x0b`P' ++ b'K\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0lH\x05\xe2' ++ b'\x1e8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00\x00\x00\x00' ++ b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00aPK' ++ b'\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0lH\x05\xe2\x1e' ++ b'8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00\x00\x00\x00\x00' ++ b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00bPK\x05' ++ b'\x06\x00\x00\x00\x00\x02\x00\x02\x00^\x00\x00\x00/\x00\x00' ++ b'\x00\x00\x00' ++ ) ++ with zipfile.ZipFile(io.BytesIO(data), 'r') as zipf: ++ self.assertEqual(zipf.namelist(), 'a', 'b') ++ zi = zipf.getinfo('a') ++ self.assertEqual(zi.header_offset, 0) ++ self.assertEqual(zi.compress_size, 16) ++ self.assertEqual(zi.file_size, 1033) ++ zi = zipf.getinfo('b') ++ self.assertEqual(zi.header_offset, 0) ++ self.assertEqual(zi.compress_size, 16) ++ self.assertEqual(zi.file_size, 1033) ++ self.assertEqual(len(zipf.read('a')), 1033) ++ with self.assertRaisesRegex(zipfile.BadZipFile, 'File name.*differ'): ++ zipf.read('b') ++ ++ @requires_zlib() ++ def test_quoted_overlap(self): ++ data = ( ++ b'PK\x03\x04\x14\x00\x00\x00\x08\x00\xa0lH\x05Y\xfc' ++ b'8\x044\x00\x00\x00(\x04\x00\x00\x01\x00\x00\x00a\x00' ++ b'\x1f\x00\xe0\xffPK\x03\x04\x14\x00\x00\x00\x08\x00\xa0l' ++ b'H\x05\xe2\x1e8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00' ++ b'\x00\x00b\xed\xc0\x81\x08\x00\x00\x00\xc00\xd6\xfbK\\' ++ b'd\x0b`PK\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0' ++ b'lH\x05Y\xfc8\x044\x00\x00\x00(\x04\x00\x00\x01' ++ b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' ++ b'\x00aPK\x01\x02\x14\x00\x14\x00\x00\x00\x08\x00\xa0l' ++ b'H\x05\xe2\x1e8\xbb\x10\x00\x00\x00\t\x04\x00\x00\x01\x00' ++ b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00$\x00\x00\x00' ++ b'bPK\x05\x06\x00\x00\x00\x00\x02\x00\x02\x00^\x00\x00' ++ b'\x00S\x00\x00\x00\x00\x00' ++ ) ++ with zipfile.ZipFile(io.BytesIO(data), 'r') as zipf: ++ self.assertEqual(zipf.namelist(), 'a', 'b') ++ zi = zipf.getinfo('a') ++ self.assertEqual(zi.header_offset, 0) ++ self.assertEqual(zi.compress_size, 52) ++ self.assertEqual(zi.file_size, 1064) ++ zi = zipf.getinfo('b') ++ self.assertEqual(zi.header_offset, 36) ++ self.assertEqual(zi.compress_size, 16) ++ self.assertEqual(zi.file_size, 1033) ++ with self.assertRaisesRegex(zipfile.BadZipFile, 'Overlapped entries'): ++ zipf.read('a') ++ self.assertEqual(len(zipf.read('b')), 1033) ++ + def tearDown(self): + unlink(TESTFN) + unlink(TESTFN2) +diff --git a/Lib/zipfile.py b/Lib/zipfile.py +index 6189db5e3e4..058d7163ea1 100644 +--- a/Lib/zipfile.py ++++ b/Lib/zipfile.py +@@ -367,6 +367,7 @@ class ZipInfo (object): + 'compress_size', + 'file_size', + '_raw_time', ++ '_end_offset', + ) + + def __init__(self, filename="NoName", date_time=(1980,1,1,0,0,0)): +@@ -408,6 +409,7 @@ def __init__(self, filename="NoName", date_time=(1980,1,1,0,0,0)): + self.external_attr = 0 # External file attributes + self.compress_size = 0 # Size of the compressed file + self.file_size = 0 # Size of the uncompressed file ++ self._end_offset = None # Start of the next local header or central directory + # Other attributes are set by class ZipFile: + # header_offset Byte offset to the file header + # CRC CRC-32 of the uncompressed file +@@ -1437,6 +1439,12 @@ def _RealGetContents(self): + if self.debug > 2: + print("total", total) + ++ end_offset = self.start_dir ++ for zinfo in sorted(self.filelist, ++ key=lambda zinfo: zinfo.header_offset, ++ reverse=True): ++ zinfo._end_offset = end_offset ++ end_offset = zinfo.header_offset + + def namelist(self): + """Return a list of file names in the archive.""" +@@ -1590,6 +1598,10 @@ def open(self, name, mode="r", pwd=None, *, force_zip64=False): + 'File name in directory %r and header %r differ.' + % (zinfo.orig_filename, fname)) + ++ if (zinfo._end_offset is not None and ++ zef_file.tell() + zinfo.compress_size > zinfo._end_offset): ++ raise BadZipFile(f"Overlapped entries: {zinfo.orig_filename!r} (possible zip bomb)") ++ + # check for encrypted flag & handle password + is_encrypted = zinfo.flag_bits & _MASK_ENCRYPTED + if is_encrypted: +diff --git a/Misc/NEWS.d/next/Library/2023-09-28-13-15-51.gh-issue-109858.43e2dg.rst b/Misc/NEWS.d/next/Library/2023-09-28-13-15-51.gh-issue-109858.43e2dg.rst +new file mode 100644 +index 00000000000..be279caffc4 +--- /dev/null ++++ b/Misc/NEWS.d/next/Library/2023-09-28-13-15-51.gh-issue-109858.43e2dg.rst +@@ -0,0 +1,3 @@ ++Protect :mod:`zipfile` from "quoted-overlap" zipbomb. It now raises ++BadZipFile when try to read an entry that overlaps with other entry or ++central directory. +-- +2.33.0 +
View file
_service:tar_scm:backport-CVE-2024-3219-1-gh-122133-Authenticate-socket-connection-for-so.patch
Added
@@ -0,0 +1,218 @@ +From 5f90abaa786f994db3907fc31e2ee00ea2cf0929 Mon Sep 17 00:00:00 2001 +From: "Miss Islington (bot)" + <31488909+miss-islington@users.noreply.github.com> +Date: Tue, 30 Jul 2024 14:43:45 +0200 +Subject: PATCH 3.11 gh-122133: Authenticate socket connection for + `socket.socketpair()` fallback (GH-122134) (#122426) + +Authenticate socket connection for `socket.socketpair()` fallback when the platform does not have a native `socketpair` C API. We authenticate in-process using `getsocketname` and `getpeername` (thanks to Nathaniel J Smith for that suggestion). + +(cherry picked from commit 78df1043dbdce5c989600616f9f87b4ee72944e5) + +Co-authored-by: Seth Michael Larson <seth@python.org> +Co-authored-by: Gregory P. Smith <greg@krypto.org> +--- + Lib/socket.py | 17 +++ + Lib/test/test_socket.py | 128 +++++++++++++++++- + ...-07-22-13-11-28.gh-issue-122133.0mPeta.rst | 5 + + 3 files changed, 147 insertions(+), 3 deletions(-) + create mode 100644 Misc/NEWS.d/next/Security/2024-07-22-13-11-28.gh-issue-122133.0mPeta.rst + +diff --git a/Lib/socket.py b/Lib/socket.py +index a0567b76bcf..591d4739a64 100644 +--- a/Lib/socket.py ++++ b/Lib/socket.py +@@ -648,6 +648,23 @@ def socketpair(family=AF_INET, type=SOCK_STREAM, proto=0): + raise + finally: + lsock.close() ++ ++ # Authenticating avoids using a connection from something else ++ # able to connect to {host}:{port} instead of us. ++ # We expect only AF_INET and AF_INET6 families. ++ try: ++ if ( ++ ssock.getsockname() != csock.getpeername() ++ or csock.getsockname() != ssock.getpeername() ++ ): ++ raise ConnectionError("Unexpected peer connection") ++ except: ++ # getsockname() and getpeername() can fail ++ # if either socket isn't connected. ++ ssock.close() ++ csock.close() ++ raise ++ + return (ssock, csock) + __all__.append("socketpair") + +diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py +index 42adc573ecc..a60eb436c7b 100644 +--- a/Lib/test/test_socket.py ++++ b/Lib/test/test_socket.py +@@ -542,19 +542,27 @@ class SocketPairTest(unittest.TestCase, ThreadableTest): + def __init__(self, methodName='runTest'): + unittest.TestCase.__init__(self, methodName=methodName) + ThreadableTest.__init__(self) ++ self.cli = None ++ self.serv = None ++ ++ def socketpair(self): ++ # To be overridden by some child classes. ++ return socket.socketpair() + + def setUp(self): +- self.serv, self.cli = socket.socketpair() ++ self.serv, self.cli = self.socketpair() + + def tearDown(self): +- self.serv.close() ++ if self.serv: ++ self.serv.close() + self.serv = None + + def clientSetUp(self): + pass + + def clientTearDown(self): +- self.cli.close() ++ if self.cli: ++ self.cli.close() + self.cli = None + ThreadableTest.clientTearDown(self) + +@@ -4667,6 +4675,120 @@ def _testSend(self): + self.assertEqual(msg, MSG) + + ++class PurePythonSocketPairTest(SocketPairTest): ++ ++ # Explicitly use socketpair AF_INET or AF_INET6 to ensure that is the ++ # code path we're using regardless platform is the pure python one where ++ # `_socket.socketpair` does not exist. (AF_INET does not work with ++ # _socket.socketpair on many platforms). ++ def socketpair(self): ++ # called by super().setUp(). ++ try: ++ return socket.socketpair(socket.AF_INET6) ++ except OSError: ++ return socket.socketpair(socket.AF_INET) ++ ++ # Local imports in this class make for easy security fix backporting. ++ ++ def setUp(self): ++ import _socket ++ self._orig_sp = getattr(_socket, 'socketpair', None) ++ if self._orig_sp is not None: ++ # This forces the version using the non-OS provided socketpair ++ # emulation via an AF_INET socket in Lib/socket.py. ++ del _socket.socketpair ++ import importlib ++ global socket ++ socket = importlib.reload(socket) ++ else: ++ pass # This platform already uses the non-OS provided version. ++ super().setUp() ++ ++ def tearDown(self): ++ super().tearDown() ++ import _socket ++ if self._orig_sp is not None: ++ # Restore the default socket.socketpair definition. ++ _socket.socketpair = self._orig_sp ++ import importlib ++ global socket ++ socket = importlib.reload(socket) ++ ++ def test_recv(self): ++ msg = self.serv.recv(1024) ++ self.assertEqual(msg, MSG) ++ ++ def _test_recv(self): ++ self.cli.send(MSG) ++ ++ def test_send(self): ++ self.serv.send(MSG) ++ ++ def _test_send(self): ++ msg = self.cli.recv(1024) ++ self.assertEqual(msg, MSG) ++ ++ def test_ipv4(self): ++ cli, srv = socket.socketpair(socket.AF_INET) ++ cli.close() ++ srv.close() ++ ++ def _test_ipv4(self): ++ pass ++ ++ @unittest.skipIf(not hasattr(_socket, 'IPPROTO_IPV6') or ++ not hasattr(_socket, 'IPV6_V6ONLY'), ++ "IPV6_V6ONLY option not supported") ++ @unittest.skipUnless(socket_helper.IPV6_ENABLED, 'IPv6 required for this test') ++ def test_ipv6(self): ++ cli, srv = socket.socketpair(socket.AF_INET6) ++ cli.close() ++ srv.close() ++ ++ def _test_ipv6(self): ++ pass ++ ++ def test_injected_authentication_failure(self): ++ orig_getsockname = socket.socket.getsockname ++ inject_sock = None ++ ++ def inject_getsocketname(self): ++ nonlocal inject_sock ++ sockname = orig_getsockname(self) ++ # Connect to the listening socket ahead of the ++ # client socket. ++ if inject_sock is None: ++ inject_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) ++ inject_sock.setblocking(False) ++ try: ++ inject_sock.connect(sockname:2) ++ except (BlockingIOError, InterruptedError): ++ pass ++ inject_sock.setblocking(True) ++ return sockname ++ ++ sock1 = sock2 = None ++ try: ++ socket.socket.getsockname = inject_getsocketname ++ with self.assertRaises(OSError): ++ sock1, sock2 = socket.socketpair() ++ finally: ++ socket.socket.getsockname = orig_getsockname ++ if inject_sock: ++ inject_sock.close() ++ if sock1: # This cleanup isn't needed on a successful test. ++ sock1.close() ++ if sock2: ++ sock2.close() ++ ++ def _test_injected_authentication_failure(self): ++ # No-op. Exists for base class threading infrastructure to call. ++ # We could refactor this test into its own lesser class along with the ++ # setUp and tearDown code to construct an ideal; it is simpler to keep ++ # it here and live with extra overhead one this _one_ failure test. ++ pass
View file
_service:tar_scm:backport-CVE-2024-3219-2-gh-122133-Rework-pure-Python-socketpair-tests-t.patch
Added
@@ -0,0 +1,207 @@ +From c5655aa6ad120d2ed7f255bebd6e8b71a9c07dde Mon Sep 17 00:00:00 2001 +From: "Miss Islington (bot)" + <31488909+miss-islington@users.noreply.github.com> +Date: Fri, 2 Aug 2024 15:09:45 +0200 +Subject: PATCH 3.11 gh-122133: Rework pure Python socketpair tests to + avoid use of importlib.reload. (GH-122493) (GH-122506) + +(cherry picked from commit f071f01b7b7e19d7d6b3a4b0ec62f820ecb14660) + +Co-authored-by: Russell Keith-Magee <russell@keith-magee.com> +Co-authored-by: Gregory P. Smith <greg@krypto.org> +--- + Lib/socket.py | 121 +++++++++++++++++++--------------------- + Lib/test/test_socket.py | 20 ++----- + 2 files changed, 64 insertions(+), 77 deletions(-) + +diff --git a/Lib/socket.py b/Lib/socket.py +index 591d4739a64..f386241abfb 100644 +--- a/Lib/socket.py ++++ b/Lib/socket.py +@@ -590,16 +590,65 @@ def fromshare(info): + return socket(0, 0, 0, info) + __all__.append("fromshare") + +-if hasattr(_socket, "socketpair"): ++# Origin: https://gist.github.com/4325783, by Geert Jansen. Public domain. ++# This is used if _socket doesn't natively provide socketpair. It's ++# always defined so that it can be patched in for testing purposes. ++def _fallback_socketpair(family=AF_INET, type=SOCK_STREAM, proto=0): ++ if family == AF_INET: ++ host = _LOCALHOST ++ elif family == AF_INET6: ++ host = _LOCALHOST_V6 ++ else: ++ raise ValueError("Only AF_INET and AF_INET6 socket address families " ++ "are supported") ++ if type != SOCK_STREAM: ++ raise ValueError("Only SOCK_STREAM socket type is supported") ++ if proto != 0: ++ raise ValueError("Only protocol zero is supported") ++ ++ # We create a connected TCP socket. Note the trick with ++ # setblocking(False) that prevents us from having to create a thread. ++ lsock = socket(family, type, proto) ++ try: ++ lsock.bind((host, 0)) ++ lsock.listen() ++ # On IPv6, ignore flow_info and scope_id ++ addr, port = lsock.getsockname():2 ++ csock = socket(family, type, proto) ++ try: ++ csock.setblocking(False) ++ try: ++ csock.connect((addr, port)) ++ except (BlockingIOError, InterruptedError): ++ pass ++ csock.setblocking(True) ++ ssock, _ = lsock.accept() ++ except: ++ csock.close() ++ raise ++ finally: ++ lsock.close() + +- def socketpair(family=None, type=SOCK_STREAM, proto=0): +- """socketpair(family, type, proto) -> (socket object, socket object) ++ # Authenticating avoids using a connection from something else ++ # able to connect to {host}:{port} instead of us. ++ # We expect only AF_INET and AF_INET6 families. ++ try: ++ if ( ++ ssock.getsockname() != csock.getpeername() ++ or csock.getsockname() != ssock.getpeername() ++ ): ++ raise ConnectionError("Unexpected peer connection") ++ except: ++ # getsockname() and getpeername() can fail ++ # if either socket isn't connected. ++ ssock.close() ++ csock.close() ++ raise + +- Create a pair of socket objects from the sockets returned by the platform +- socketpair() function. +- The arguments are the same as for socket() except the default family is +- AF_UNIX if defined on the platform; otherwise, the default is AF_INET. +- """ ++ return (ssock, csock) ++ ++if hasattr(_socket, "socketpair"): ++ def socketpair(family=None, type=SOCK_STREAM, proto=0): + if family is None: + try: + family = AF_UNIX +@@ -611,61 +660,7 @@ def socketpair(family=None, type=SOCK_STREAM, proto=0): + return a, b + + else: +- +- # Origin: https://gist.github.com/4325783, by Geert Jansen. Public domain. +- def socketpair(family=AF_INET, type=SOCK_STREAM, proto=0): +- if family == AF_INET: +- host = _LOCALHOST +- elif family == AF_INET6: +- host = _LOCALHOST_V6 +- else: +- raise ValueError("Only AF_INET and AF_INET6 socket address families " +- "are supported") +- if type != SOCK_STREAM: +- raise ValueError("Only SOCK_STREAM socket type is supported") +- if proto != 0: +- raise ValueError("Only protocol zero is supported") +- +- # We create a connected TCP socket. Note the trick with +- # setblocking(False) that prevents us from having to create a thread. +- lsock = socket(family, type, proto) +- try: +- lsock.bind((host, 0)) +- lsock.listen() +- # On IPv6, ignore flow_info and scope_id +- addr, port = lsock.getsockname():2 +- csock = socket(family, type, proto) +- try: +- csock.setblocking(False) +- try: +- csock.connect((addr, port)) +- except (BlockingIOError, InterruptedError): +- pass +- csock.setblocking(True) +- ssock, _ = lsock.accept() +- except: +- csock.close() +- raise +- finally: +- lsock.close() +- +- # Authenticating avoids using a connection from something else +- # able to connect to {host}:{port} instead of us. +- # We expect only AF_INET and AF_INET6 families. +- try: +- if ( +- ssock.getsockname() != csock.getpeername() +- or csock.getsockname() != ssock.getpeername() +- ): +- raise ConnectionError("Unexpected peer connection") +- except: +- # getsockname() and getpeername() can fail +- # if either socket isn't connected. +- ssock.close() +- csock.close() +- raise +- +- return (ssock, csock) ++ socketpair = _fallback_socketpair + __all__.append("socketpair") + + socketpair.__doc__ = """socketpair(family, type, proto) -> (socket object, socket object) +diff --git a/Lib/test/test_socket.py b/Lib/test/test_socket.py +index a60eb436c7b..cc803d8753b 100644 +--- a/Lib/test/test_socket.py ++++ b/Lib/test/test_socket.py +@@ -4676,7 +4676,6 @@ def _testSend(self): + + + class PurePythonSocketPairTest(SocketPairTest): +- + # Explicitly use socketpair AF_INET or AF_INET6 to ensure that is the + # code path we're using regardless platform is the pure python one where + # `_socket.socketpair` does not exist. (AF_INET does not work with +@@ -4691,28 +4690,21 @@ def socketpair(self): + # Local imports in this class make for easy security fix backporting. + + def setUp(self): +- import _socket +- self._orig_sp = getattr(_socket, 'socketpair', None) +- if self._orig_sp is not None: ++ if hasattr(_socket, "socketpair"): ++ self._orig_sp = socket.socketpair + # This forces the version using the non-OS provided socketpair + # emulation via an AF_INET socket in Lib/socket.py. +- del _socket.socketpair +- import importlib +- global socket +- socket = importlib.reload(socket) ++ socket.socketpair = socket._fallback_socketpair + else: +- pass # This platform already uses the non-OS provided version. ++ # This platform already uses the non-OS provided version. ++ self._orig_sp = None + super().setUp() + + def tearDown(self): + super().tearDown() +- import _socket + if self._orig_sp is not None: + # Restore the default socket.socketpair definition. +- _socket.socketpair = self._orig_sp +- import importlib +- global socket
View file
_service:tar_scm:backport-CVE-2024-4032-gh-113171-gh-65056-Fix-private-non-global-IP-ad.patch
Added
@@ -0,0 +1,346 @@ +From ba431579efdcbaed7a96f2ac4ea0775879a332fb Mon Sep 17 00:00:00 2001 +From: Petr Viktorin <encukou@gmail.com> +Date: Thu, 25 Apr 2024 14:45:48 +0200 +Subject: PATCH 3.11 gh-113171: gh-65056: Fix "private" (non-global) IP + address ranges (GH-113179) (GH-113186) (GH-118177) (#118227) + +--- + Doc/library/ipaddress.rst | 43 +++++++- + Doc/whatsnew/3.11.rst | 9 ++ + Lib/ipaddress.py | 99 +++++++++++++++---- + Lib/test/test_ipaddress.py | 21 +++- + ...-03-14-01-38-44.gh-issue-113171.VFnObz.rst | 9 ++ + 5 files changed, 157 insertions(+), 24 deletions(-) + create mode 100644 Misc/NEWS.d/next/Library/2024-03-14-01-38-44.gh-issue-113171.VFnObz.rst + +diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst +index 03dc956cd1..f57fa15aa5 100644 +--- a/Doc/library/ipaddress.rst ++++ b/Doc/library/ipaddress.rst +@@ -178,18 +178,53 @@ write code that handles both IP versions correctly. Address objects are + + .. attribute:: is_private + +- ``True`` if the address is allocated for private networks. See ++ ``True`` if the address is defined as not globally reachable by + iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ +- (for IPv6). ++ (for IPv6) with the following exceptions: ++ ++ * ``is_private`` is ``False`` for the shared address space (``100.64.0.0/10``) ++ * For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the ++ semantics of the underlying IPv4 addresses and the following condition holds ++ (see :attr:`IPv6Address.ipv4_mapped`):: ++ ++ address.is_private == address.ipv4_mapped.is_private ++ ++ ``is_private`` has value opposite to :attr:`is_global`, except for the shared address space ++ (``100.64.0.0/10`` range) where they are both ``False``. ++ ++ .. versionchanged:: 3.11.10 ++ ++ Fixed some false positives and false negatives. ++ ++ * ``192.0.0.0/24`` is considered private with the exception of ``192.0.0.9/32`` and ++ ``192.0.0.10/32`` (previously: only the ``192.0.0.0/29`` sub-range was considered private). ++ * ``64:ff9b:1::/48`` is considered private. ++ * ``2002::/16`` is considered private. ++ * There are exceptions within ``2001::/23`` (otherwise considered private): ``2001:1::1/128``, ++ ``2001:1::2/128``, ``2001:3::/32``, ``2001:4:112::/48``, ``2001:20::/28``, ``2001:30::/28``. ++ The exceptions are not considered private. + + .. attribute:: is_global + +- ``True`` if the address is allocated for public networks. See ++ ``True`` if the address is defined as globally reachable by + iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ +- (for IPv6). ++ (for IPv6) with the following exception: ++ ++ For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the ++ semantics of the underlying IPv4 addresses and the following condition holds ++ (see :attr:`IPv6Address.ipv4_mapped`):: ++ ++ address.is_global == address.ipv4_mapped.is_global ++ ++ ``is_global`` has value opposite to :attr:`is_private`, except for the shared address space ++ (``100.64.0.0/10`` range) where they are both ``False``. + + .. versionadded:: 3.4 + ++ .. versionchanged:: 3.11.10 ++ ++ Fixed some false positives and false negatives, see :attr:`is_private` for details. ++ + .. attribute:: is_unspecified + + ``True`` if the address is unspecified. See :RFC:`5735` (for IPv4) +diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst +index f670fa1f09..42b61c75c7 100644 +--- a/Doc/whatsnew/3.11.rst ++++ b/Doc/whatsnew/3.11.rst +@@ -2727,3 +2727,12 @@ OpenSSL + * Windows builds and macOS installers from python.org now use OpenSSL 3.0. + + .. _libb2: https://www.blake2.net/ ++ ++Notable changes in 3.11.10 ++========================== ++ ++ipaddress ++--------- ++ ++* Fixed ``is_global`` and ``is_private`` behavior in ``IPv4Address``, ++ ``IPv6Address``, ``IPv4Network`` and ``IPv6Network``. +diff --git a/Lib/ipaddress.py b/Lib/ipaddress.py +index 16ba16cd7d..567beb37e0 100644 +--- a/Lib/ipaddress.py ++++ b/Lib/ipaddress.py +@@ -1086,7 +1086,11 @@ def is_private(self): + """ + return any(self.network_address in priv_network and + self.broadcast_address in priv_network +- for priv_network in self._constants._private_networks) ++ for priv_network in self._constants._private_networks) and all( ++ self.network_address not in network and ++ self.broadcast_address not in network ++ for network in self._constants._private_networks_exceptions ++ ) + + @property + def is_global(self): +@@ -1333,18 +1337,41 @@ def is_reserved(self): + @property + @functools.lru_cache() + def is_private(self): +- """Test if this address is allocated for private networks. ++ """``True`` if the address is defined as not globally reachable by ++ iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ ++ (for IPv6) with the following exceptions: + +- Returns: +- A boolean, True if the address is reserved per +- iana-ipv4-special-registry. ++ * ``is_private`` is ``False`` for ``100.64.0.0/10`` ++ * For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the ++ semantics of the underlying IPv4 addresses and the following condition holds ++ (see :attr:`IPv6Address.ipv4_mapped`):: + ++ address.is_private == address.ipv4_mapped.is_private ++ ++ ``is_private`` has value opposite to :attr:`is_global`, except for the ``100.64.0.0/10`` ++ IPv4 range where they are both ``False``. + """ +- return any(self in net for net in self._constants._private_networks) ++ return ( ++ any(self in net for net in self._constants._private_networks) ++ and all(self not in net for net in self._constants._private_networks_exceptions) ++ ) + + @property + @functools.lru_cache() + def is_global(self): ++ """``True`` if the address is defined as globally reachable by ++ iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ ++ (for IPv6) with the following exception: ++ ++ For IPv4-mapped IPv6-addresses the ``is_private`` value is determined by the ++ semantics of the underlying IPv4 addresses and the following condition holds ++ (see :attr:`IPv6Address.ipv4_mapped`):: ++ ++ address.is_global == address.ipv4_mapped.is_global ++ ++ ``is_global`` has value opposite to :attr:`is_private`, except for the ``100.64.0.0/10`` ++ IPv4 range where they are both ``False``. ++ """ + return self not in self._constants._public_network and not self.is_private + + @property +@@ -1548,13 +1575,15 @@ class _IPv4Constants: + + _public_network = IPv4Network('100.64.0.0/10') + ++ # Not globally reachable address blocks listed on ++ # https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml + _private_networks = + IPv4Network('0.0.0.0/8'), + IPv4Network('10.0.0.0/8'), + IPv4Network('127.0.0.0/8'), + IPv4Network('169.254.0.0/16'), + IPv4Network('172.16.0.0/12'), +- IPv4Network('192.0.0.0/29'), ++ IPv4Network('192.0.0.0/24'), + IPv4Network('192.0.0.170/31'), + IPv4Network('192.0.2.0/24'), + IPv4Network('192.168.0.0/16'), +@@ -1565,6 +1594,11 @@ class _IPv4Constants: + IPv4Network('255.255.255.255/32'), + + ++ _private_networks_exceptions = ++ IPv4Network('192.0.0.9/32'), ++ IPv4Network('192.0.0.10/32'), ++ ++ + _reserved_network = IPv4Network('240.0.0.0/4') + + _unspecified_address = IPv4Address('0.0.0.0') +@@ -2010,27 +2044,42 @@ def is_site_local(self): + @property + @functools.lru_cache() + def is_private(self): +- """Test if this address is allocated for private networks. ++ """``True`` if the address is defined as not globally reachable by ++ iana-ipv4-special-registry_ (for IPv4) or iana-ipv6-special-registry_ ++ (for IPv6) with the following exceptions: + +- Returns: +- A boolean, True if the address is reserved per +- iana-ipv6-special-registry, or is ipv4_mapped and is
View file
_service:tar_scm:backport-CVE-2024-6232-gh-121285-Remove-backtracking-when-parsing-tarf.patch
Added
@@ -0,0 +1,247 @@ +From d449caf8a179e3b954268b3a88eb9170be3c8fbf Mon Sep 17 00:00:00 2001 +From: Seth Michael Larson <seth@python.org> +Date: Tue, 3 Sep 2024 10:07:13 -0500 +Subject: PATCH 3.11 gh-121285: Remove backtracking when parsing tarfile + headers (GH-121286) (#123639) + +* Remove backtracking when parsing tarfile headers +* Rewrite PAX header parsing to be stricter +* Optimize parsing of GNU extended sparse headers v0.0 + +(cherry picked from commit 34ddb64d088dd7ccc321f6103d23153256caa5d4) + +Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru> +Co-authored-by: Gregory P. Smith <greg@krypto.org> +--- + Lib/tarfile.py | 105 +++++++++++------- + Lib/test/test_tarfile.py | 42 +++++++ + ...-07-02-13-39-20.gh-issue-121285.hrl-yI.rst | 2 + + 3 files changed, 111 insertions(+), 38 deletions(-) + create mode 100644 Misc/NEWS.d/next/Security/2024-07-02-13-39-20.gh-issue-121285.hrl-yI.rst + +diff --git a/Lib/tarfile.py b/Lib/tarfile.py +index 612217b1ad0..0d6b925533b 100755 +--- a/Lib/tarfile.py ++++ b/Lib/tarfile.py +@@ -842,6 +842,9 @@ def data_filter(member, dest_path): + # Sentinel for replace() defaults, meaning "don't change the attribute" + _KEEP = object() + ++# Header length is digits followed by a space. ++_header_length_prefix_re = re.compile(br"(0-9{1,20}) ") ++ + class TarInfo(object): + """Informational class which holds the details about an + archive member given by a tar header block. +@@ -1411,41 +1414,59 @@ def _proc_pax(self, tarfile): + else: + pax_headers = tarfile.pax_headers.copy() + +- # Check if the pax header contains a hdrcharset field. This tells us +- # the encoding of the path, linkpath, uname and gname fields. Normally, +- # these fields are UTF-8 encoded but since POSIX.1-2008 tar +- # implementations are allowed to store them as raw binary strings if +- # the translation to UTF-8 fails. +- match = re.search(br"\d+ hdrcharset=(^\n+)\n", buf) +- if match is not None: +- pax_headers"hdrcharset" = match.group(1).decode("utf-8") +- +- # For the time being, we don't care about anything other than "BINARY". +- # The only other value that is currently allowed by the standard is +- # "ISO-IR 10646 2000 UTF-8" in other words UTF-8. +- hdrcharset = pax_headers.get("hdrcharset") +- if hdrcharset == "BINARY": +- encoding = tarfile.encoding +- else: +- encoding = "utf-8" +- + # Parse pax header information. A record looks like that: + # "%d %s=%s\n" % (length, keyword, value). length is the size + # of the complete record including the length field itself and +- # the newline. keyword and value are both UTF-8 encoded strings. +- regex = re.compile(br"(\d+) (^=+)=") ++ # the newline. + pos = 0 +- while True: +- match = regex.match(buf, pos) +- if not match: +- break ++ encoding = None ++ raw_headers = ++ while len(buf) > pos and bufpos != 0x00: ++ if not (match := _header_length_prefix_re.match(buf, pos)): ++ raise InvalidHeaderError("invalid header") ++ try: ++ length = int(match.group(1)) ++ except ValueError: ++ raise InvalidHeaderError("invalid header") ++ # Headers must be at least 5 bytes, shortest being '5 x=\n'. ++ # Value is allowed to be empty. ++ if length < 5: ++ raise InvalidHeaderError("invalid header") ++ if pos + length > len(buf): ++ raise InvalidHeaderError("invalid header") + +- length, keyword = match.groups() +- length = int(length) +- if length == 0: ++ header_value_end_offset = match.start(1) + length - 1 # Last byte of the header ++ keyword_and_value = bufmatch.end(1) + 1:header_value_end_offset ++ raw_keyword, equals, raw_value = keyword_and_value.partition(b"=") ++ ++ # Check the framing of the header. The last character must be '\n' (0x0A) ++ if not raw_keyword or equals != b"=" or bufheader_value_end_offset != 0x0A: + raise InvalidHeaderError("invalid header") +- value = bufmatch.end(2) + 1:match.start(1) + length - 1 ++ raw_headers.append((length, raw_keyword, raw_value)) ++ ++ # Check if the pax header contains a hdrcharset field. This tells us ++ # the encoding of the path, linkpath, uname and gname fields. Normally, ++ # these fields are UTF-8 encoded but since POSIX.1-2008 tar ++ # implementations are allowed to store them as raw binary strings if ++ # the translation to UTF-8 fails. For the time being, we don't care about ++ # anything other than "BINARY". The only other value that is currently ++ # allowed by the standard is "ISO-IR 10646 2000 UTF-8" in other words UTF-8. ++ # Note that we only follow the initial 'hdrcharset' setting to preserve ++ # the initial behavior of the 'tarfile' module. ++ if raw_keyword == b"hdrcharset" and encoding is None: ++ if raw_value == b"BINARY": ++ encoding = tarfile.encoding ++ else: # This branch ensures only the first 'hdrcharset' header is used. ++ encoding = "utf-8" ++ ++ pos += length + ++ # If no explicit hdrcharset is set, we use UTF-8 as a default. ++ if encoding is None: ++ encoding = "utf-8" ++ ++ # After parsing the raw headers we can decode them to text. ++ for length, raw_keyword, raw_value in raw_headers: + # Normally, we could just use "utf-8" as the encoding and "strict" + # as the error handler, but we better not take the risk. For + # example, GNU tar <= 1.23 is known to store filenames it cannot +@@ -1453,17 +1474,16 @@ def _proc_pax(self, tarfile): + # hdrcharset=BINARY header). + # We first try the strict standard encoding, and if that fails we + # fall back on the user's encoding and error handler. +- keyword = self._decode_pax_field(keyword, "utf-8", "utf-8", ++ keyword = self._decode_pax_field(raw_keyword, "utf-8", "utf-8", + tarfile.errors) + if keyword in PAX_NAME_FIELDS: +- value = self._decode_pax_field(value, encoding, tarfile.encoding, ++ value = self._decode_pax_field(raw_value, encoding, tarfile.encoding, + tarfile.errors) + else: +- value = self._decode_pax_field(value, "utf-8", "utf-8", ++ value = self._decode_pax_field(raw_value, "utf-8", "utf-8", + tarfile.errors) + + pax_headerskeyword = value +- pos += length + + # Fetch the next header. + try: +@@ -1478,7 +1498,7 @@ def _proc_pax(self, tarfile): + + elif "GNU.sparse.size" in pax_headers: + # GNU extended sparse format version 0.0. +- self._proc_gnusparse_00(next, pax_headers, buf) ++ self._proc_gnusparse_00(next, raw_headers) + + elif pax_headers.get("GNU.sparse.major") == "1" and pax_headers.get("GNU.sparse.minor") == "0": + # GNU extended sparse format version 1.0. +@@ -1500,15 +1520,24 @@ def _proc_pax(self, tarfile): + + return next + +- def _proc_gnusparse_00(self, next, pax_headers, buf): ++ def _proc_gnusparse_00(self, next, raw_headers): + """Process a GNU tar extended sparse header, version 0.0. + """ + offsets = +- for match in re.finditer(br"\d+ GNU.sparse.offset=(\d+)\n", buf): +- offsets.append(int(match.group(1))) + numbytes = +- for match in re.finditer(br"\d+ GNU.sparse.numbytes=(\d+)\n", buf): +- numbytes.append(int(match.group(1))) ++ for _, keyword, value in raw_headers: ++ if keyword == b"GNU.sparse.offset": ++ try: ++ offsets.append(int(value.decode())) ++ except ValueError: ++ raise InvalidHeaderError("invalid header") ++ ++ elif keyword == b"GNU.sparse.numbytes": ++ try: ++ numbytes.append(int(value.decode())) ++ except ValueError: ++ raise InvalidHeaderError("invalid header") ++ + next.sparse = list(zip(offsets, numbytes)) + + def _proc_gnusparse_01(self, next, pax_headers): +diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py +index 389da7be3a3..c99c88ce93a 100644 +--- a/Lib/test/test_tarfile.py ++++ b/Lib/test/test_tarfile.py +@@ -1208,6 +1208,48 @@ def test_pax_number_fields(self): + finally: + tar.close() + ++ def test_pax_header_bad_formats(self): ++ # The fields from the pax header have priority over the ++ # TarInfo. ++ pax_header_replacements = ( ++ b" foo=bar\n", ++ b"0 \n", ++ b"1 \n", ++ b"2 \n",
View file
_service:tar_scm:backport-CVE-2024-6923-gh-121650-Encode-newlines-in-headers-and-verify-head.patch
Added
@@ -0,0 +1,348 @@ +From 67067f7cca97cc980d31c5769f621b58ef876b7e Mon Sep 17 00:00:00 2001 +From: xinsheng <xinsheng3@huawei.com> +Date: Tue, 3 Sep 2024 21:17:26 +0800 +Subject: PATCH gh-121650: Encode newlines in headers, and verify headers are + sound (GH-122233) + +--- + Doc/library/email.errors.rst | 6 ++ + Doc/library/email.policy.rst | 18 ++++++ + Doc/whatsnew/3.11.rst | 13 ++++ + Lib/email/_header_value_parser.py | 12 +++- + Lib/email/_policybase.py | 8 +++ + Lib/email/errors.py | 4 ++ + Lib/email/generator.py | 13 +++- + Lib/test/test_email/test_generator.py | 62 +++++++++++++++++++ + Lib/test/test_email/test_policy.py | 26 ++++++++ + ...-07-27-16-10-41.gh-issue-121650.nf6oc9.rst | 5 ++ + 10 files changed, 163 insertions(+), 4 deletions(-) + create mode 100644 Misc/NEWS.d/next/Library/2024-07-27-16-10-41.gh-issue-121650.nf6oc9.rst + +diff --git a/Doc/library/email.errors.rst b/Doc/library/email.errors.rst +index 194a986..f737f02 100644 +--- a/Doc/library/email.errors.rst ++++ b/Doc/library/email.errors.rst +@@ -59,6 +59,12 @@ The following exception classes are defined in the :mod:`email.errors` module: + :class:`~email.mime.image.MIMEImage`). + + ++.. exception:: HeaderWriteError() ++ ++ Raised when an error occurs when the :mod:`~email.generator` outputs ++ headers. ++ ++ + Here is the list of the defects that the :class:`~email.parser.FeedParser` + can find while parsing messages. Note that the defects are added to the message + where the problem was found, so for example, if a message nested inside a +diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst +index bf53b95..79b1786 100644 +--- a/Doc/library/email.policy.rst ++++ b/Doc/library/email.policy.rst +@@ -229,6 +229,24 @@ added matters. To illustrate:: + + .. versionadded:: 3.6 + ++ ++ .. attribute:: verify_generated_headers ++ ++ If ``True`` (the default), the generator will raise ++ :exc:`~email.errors.HeaderWriteError` instead of writing a header ++ that is improperly folded or delimited, such that it would ++ be parsed as multiple headers or joined with adjacent data. ++ Such headers can be generated by custom header classes or bugs ++ in the ``email`` module. ++ ++ As it's a security feature, this defaults to ``True`` even in the ++ :class:`~email.policy.Compat32` policy. ++ For backwards compatible, but unsafe, behavior, it must be set to ++ ``False`` explicitly. ++ ++ .. versionadded:: 3.11.10 ++ ++ + The following :class:`Policy` method is intended to be called by code using + the email library to create policy instances with custom settings: + +diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst +index c47c895..d46df98 100644 +--- a/Doc/whatsnew/3.11.rst ++++ b/Doc/whatsnew/3.11.rst +@@ -2727,6 +2727,7 @@ OpenSSL + + .. _libb2: https://www.blake2.net/ + ++ + Notable changes in 3.11.10 + ========================== + +@@ -2735,3 +2736,15 @@ ipaddress + + * Fixed ``is_global`` and ``is_private`` behavior in ``IPv4Address``, + ``IPv6Address``, ``IPv4Network`` and ``IPv6Network``. ++ ++email ++----- ++ ++* Headers with embedded newlines are now quoted on output. ++ ++ The :mod:`~email.generator` will now refuse to serialize (write) headers ++ that are improperly folded or delimited, such that they would be parsed as ++ multiple headers or joined with adjacent data. ++ If you need to turn this safety feature off, ++ set :attr:`~email.policy.Policy.verify_generated_headers`. ++ (Contributed by Bas Bloemsaat and Petr Viktorin in :gh:`121650`.) +\ No newline at end of file +diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py +index e637e6d..e1b99d5 100644 +--- a/Lib/email/_header_value_parser.py ++++ b/Lib/email/_header_value_parser.py +@@ -92,6 +92,8 @@ TOKEN_ENDS = TSPECIALS | WSP + ASPECIALS = TSPECIALS | set("*'%") + ATTRIBUTE_ENDS = ASPECIALS | WSP + EXTENDED_ATTRIBUTE_ENDS = ATTRIBUTE_ENDS - set('%') ++NLSET = {'\n', '\r'} ++SPECIALSNL = SPECIALS | NLSET + + def quote_string(value): + return '"'+str(value).replace('\\', '\\\\').replace('"', r'\"')+'"' +@@ -2778,9 +2780,13 @@ def _refold_parse_tree(parse_tree, *, policy): + wrap_as_ew_blocked -= 1 + continue + tstr = str(part) +- if part.token_type == 'ptext' and set(tstr) & SPECIALS: +- # Encode if tstr contains special characters. +- want_encoding = True ++ if not want_encoding: ++ if part.token_type == 'ptext': ++ # Encode if tstr contains special characters. ++ want_encoding = not SPECIALSNL.isdisjoint(tstr) ++ else: ++ # Encode if tstr contains newlines. ++ want_encoding = not NLSET.isdisjoint(tstr) + try: + tstr.encode(encoding) + charset = encoding +diff --git a/Lib/email/_policybase.py b/Lib/email/_policybase.py +index c9cbadd..d1f4821 100644 +--- a/Lib/email/_policybase.py ++++ b/Lib/email/_policybase.py +@@ -157,6 +157,13 @@ class Policy(_PolicyBase, metaclass=abc.ABCMeta): + message_factory -- the class to use to create new message objects. + If the value is None, the default is Message. + ++ verify_generated_headers ++ -- if true, the generator verifies that each header ++ they are properly folded, so that a parser won't ++ treat it as multiple headers, start-of-body, or ++ part of another header. ++ This is a check against custom Header & fold() ++ implementations. + """ + + raise_on_defect = False +@@ -165,6 +172,7 @@ class Policy(_PolicyBase, metaclass=abc.ABCMeta): + max_line_length = 78 + mangle_from_ = False + message_factory = None ++ verify_generated_headers = True + + def handle_defect(self, obj, defect): + """Based on policy, either raise defect or call register_defect. +diff --git a/Lib/email/errors.py b/Lib/email/errors.py +index 3ad0056..02aa5ec 100644 +--- a/Lib/email/errors.py ++++ b/Lib/email/errors.py +@@ -29,6 +29,10 @@ class CharsetError(MessageError): + """An illegal charset was given.""" + + ++class HeaderWriteError(MessageError): ++ """Error while writing headers.""" ++ ++ + # These are parsing defects which the parser was able to work around. + class MessageDefect(ValueError): + """Base class for a message defect.""" +diff --git a/Lib/email/generator.py b/Lib/email/generator.py +index b8c1091..9cdd95f 100644 +--- a/Lib/email/generator.py ++++ b/Lib/email/generator.py +@@ -14,12 +14,14 @@ import random + from copy import deepcopy + from io import StringIO, BytesIO + from email.utils import _has_surrogates ++from email.errors import HeaderWriteError + + UNDERSCORE = '_' + NL = '\n' # XXX: no longer used by the code below. + + NLCRE = re.compile(r'\r\n|\r|\n') + fcre = re.compile(r'^From ', re.MULTILINE) ++NEWLINE_WITHOUT_FWSP = re.compile(r'\r\n^ \t|\r^ \n\t|\n^ \t') + + + class Generator: +@@ -222,7 +224,16 @@ class Generator: + + def _write_headers(self, msg): + for h, v in msg.raw_items(): +- self.write(self.policy.fold(h, v)) ++ folded = self.policy.fold(h, v) ++ if self.policy.verify_generated_headers: ++ linesep = self.policy.linesep ++ if not folded.endswith(self.policy.linesep): ++ raise HeaderWriteError( ++ f'folded header does not end with {linesep!r}: {folded!r}') ++ if NEWLINE_WITHOUT_FWSP.search(folded.removesuffix(linesep)): ++ raise HeaderWriteError( ++ f'folded header contains newline: {folded!r}')
View file
_service:tar_scm:backport-CVE-2024-7592-gh-123067-Fix-quadratic-complexity-in-parsing-quoted.patch
Added
@@ -0,0 +1,131 @@ +From cc22eb02afab967715da592536fc5eeab7d7c765 Mon Sep 17 00:00:00 2001 +From: xinsheng <xinsheng3@huawei.com> +Date: Tue, 3 Sep 2024 14:58:02 +0800 +Subject: PATCH gh-123067: Fix quadratic complexity in parsing ^Cuoted cookie + values with backslashes + +--- + Lib/http/cookies.py | 34 ++++------------- + Lib/test/test_http_cookies.py | 38 +++++++++++++++++++ + ...-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst | 1 + + 3 files changed, 47 insertions(+), 26 deletions(-) + create mode 100644 Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst + +diff --git a/Lib/http/cookies.py b/Lib/http/cookies.py +index 35ac2dc..2c1f021 100644 +--- a/Lib/http/cookies.py ++++ b/Lib/http/cookies.py +@@ -184,8 +184,13 @@ def _quote(str): + return '"' + str.translate(_Translator) + '"' + + +-_OctalPatt = re.compile(r"\\0-30-70-7") +-_QuotePatt = re.compile(r"\\.") ++_unquote_sub = re.compile(r'\\(?:(0-30-70-7)|(.))').sub ++ ++def _unquote_replace(m): ++ if m1: ++ return chr(int(m1, 8)) ++ else: ++ return m2 + + def _unquote(str): + # If there aren't any doublequotes, +@@ -205,30 +210,7 @@ def _unquote(str): + # \012 --> \n + # \" --> " + # +- i = 0 +- n = len(str) +- res = +- while 0 <= i < n: +- o_match = _OctalPatt.search(str, i) +- q_match = _QuotePatt.search(str, i) +- if not o_match and not q_match: # Neither matched +- res.append(stri:) +- break +- # else: +- j = k = -1 +- if o_match: +- j = o_match.start(0) +- if q_match: +- k = q_match.start(0) +- if q_match and (not o_match or k < j): # QuotePatt matched +- res.append(stri:k) +- res.append(strk+1) +- i = k + 2 +- else: # OctalPatt matched +- res.append(stri:j) +- res.append(chr(int(strj+1:j+4, 8))) +- i = j + 4 +- return _nulljoin(res) ++ return _unquote_sub(_unquote_replace, str) + + # The _getdate() routine is used to set the expiration time in the cookie's HTTP + # header. By default, _getdate() returns the current time in the appropriate +diff --git a/Lib/test/test_http_cookies.py b/Lib/test/test_http_cookies.py +index 925c869..8879902 100644 +--- a/Lib/test/test_http_cookies.py ++++ b/Lib/test/test_http_cookies.py +@@ -5,6 +5,7 @@ import unittest + import doctest + from http import cookies + import pickle ++from test import support + + + class CookieTests(unittest.TestCase): +@@ -58,6 +59,43 @@ class CookieTests(unittest.TestCase): + for k, v in sorted(case'dict'.items()): + self.assertEqual(Ck.value, v) + ++ def test_unquote(self): ++ cases = ++ (r'a="b=\""', 'b="'), ++ (r'a="b=\\"', 'b=\\'), ++ (r'a="b=\="', 'b=='), ++ (r'a="b=\n"', 'b=n'), ++ (r'a="b=\042"', 'b="'), ++ (r'a="b=\134"', 'b=\\'), ++ (r'a="b=\377"', 'b=\xff'), ++ (r'a="b=\400"', 'b=400'), ++ (r'a="b=\42"', 'b=42'), ++ (r'a="b=\\042"', 'b=\\042'), ++ (r'a="b=\\134"', 'b=\\134'), ++ (r'a="b=\\\""', 'b=\\"'), ++ (r'a="b=\\\042"', 'b=\\"'), ++ (r'a="b=\134\""', 'b=\\"'), ++ (r'a="b=\134\042"', 'b=\\"'), ++ ++ for encoded, decoded in cases: ++ with self.subTest(encoded): ++ C = cookies.SimpleCookie() ++ C.load(encoded) ++ self.assertEqual(C'a'.value, decoded) ++ ++ @support.requires_resource('cpu') ++ def test_unquote_large(self): ++ n = 10**6 ++ for encoded in r'\\', r'\134': ++ with self.subTest(encoded): ++ data = 'a="b=' + encoded*n + ';"' ++ C = cookies.SimpleCookie() ++ C.load(data) ++ value = C'a'.value ++ self.assertEqual(value:3, 'b=\\') ++ self.assertEqual(value-2:, '\\;') ++ self.assertEqual(len(value), n + 3) ++ + def test_load(self): + C = cookies.SimpleCookie() + C.load('Customer="WILE_E_COYOTE"; Version=1; Path=/acme') +diff --git a/Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst b/Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst +new file mode 100644 +index 0000000..6a23456 +--- /dev/null ++++ b/Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst +@@ -0,0 +1 @@ ++Fix quadratic complexity in parsing ``"``-quoted cookie values with backslashes by :mod:`http.cookies`. +-- +2.43.0 +
View file
_service:tar_scm:backport-CVE-2024-8088-gh-123270-Replaced-SanitizedNames-with-a-more-surgic.patch
Added
@@ -0,0 +1,150 @@ +From 148c957b6f1fe22ed5465f4b05adb87db361fb0f Mon Sep 17 00:00:00 2001 +From: xinsheng <xinsheng3@huawei.com> +Date: Wed, 4 Sep 2024 19:21:41 +0800 +Subject: PATCH gh-123270: Replaced SanitizedNames with a more surgical fix. + (GH-123354) + +--- + Lib/test/test_zipfile.py | 77 +++++++++++++++++++ + Lib/zipfile.py | 10 ++- + ...-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst | 3 + + 3 files changed, 88 insertions(+), 2 deletions(-) + create mode 100644 Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst + +diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py +index c8e0159..669e494 100644 +--- a/Lib/test/test_zipfile.py ++++ b/Lib/test/test_zipfile.py +@@ -3512,6 +3512,83 @@ with zipfile.ZipFile(io.BytesIO(), "w") as zf: + zipfile.Path(zf) + zf.extractall(source_path.parent) + ++ def test_malformed_paths(self): ++ """ ++ Path should handle malformed paths gracefully. ++ ++ Paths with leading slashes are not visible. ++ ++ Paths with dots are treated like regular files. ++ """ ++ data = io.BytesIO() ++ zf = zipfile.ZipFile(data, "w") ++ zf.writestr("/one-slash.txt", b"content") ++ zf.writestr("//two-slash.txt", b"content") ++ zf.writestr("../parent.txt", b"content") ++ zf.filename = '' ++ root = zipfile.Path(zf) ++ assert list(map(str, root.iterdir())) == '../' ++ assert root.joinpath('..').joinpath('parent.txt').read_bytes() == b'content' ++ ++ def test_unsupported_names(self): ++ """ ++ Path segments with special characters are readable. ++ ++ On some platforms or file systems, characters like ++ ``:`` and ``?`` are not allowed, but they are valid ++ in the zip file. ++ """ ++ data = io.BytesIO() ++ zf = zipfile.ZipFile(data, "w") ++ zf.writestr("path?", b"content") ++ zf.writestr("V: NMS.flac", b"fLaC...") ++ zf.filename = '' ++ root = zipfile.Path(zf) ++ contents = root.iterdir() ++ assert next(contents).name == 'path?' ++ assert next(contents).name == 'V: NMS.flac' ++ assert root.joinpath('V: NMS.flac').read_bytes() == b"fLaC..." ++ ++ def test_backslash_not_separator(self): ++ """ ++ In a zip file, backslashes are not separators. ++ """ ++ data = io.BytesIO() ++ zf = zipfile.ZipFile(data, "w") ++ zf.writestr(DirtyZipInfo.for_name("foo\\bar", zf), b"content") ++ zf.filename = '' ++ root = zipfile.Path(zf) ++ (first,) = root.iterdir() ++ assert not first.is_dir() ++ assert first.name == 'foo\\bar' ++ ++ ++class DirtyZipInfo(zipfile.ZipInfo): ++ """ ++ Bypass name sanitization. ++ """ ++ ++ def __init__(self, filename, *args, **kwargs): ++ super().__init__(filename, *args, **kwargs) ++ self.filename = filename ++ ++ @classmethod ++ def for_name(cls, name, archive): ++ """ ++ Construct the same way that ZipFile.writestr does. ++ ++ TODO: extract this functionality and re-use ++ """ ++ self = cls(filename=name, date_time=time.localtime(time.time()):6) ++ self.compress_type = archive.compression ++ self.compress_level = archive.compresslevel ++ if self.filename.endswith('/'): # pragma: no cover ++ self.external_attr = 0o40775 << 16 # drwxrwxr-x ++ self.external_attr |= 0x10 # MS-DOS directory flag ++ else: ++ self.external_attr = 0o600 << 16 # ?rw------- ++ return self ++ + + class EncodedMetadataTests(unittest.TestCase): + file_names = '\u4e00', '\u4e8c', '\u4e09' # Han 'one', 'two', 'three' +diff --git a/Lib/zipfile.py b/Lib/zipfile.py +index 6189db5..622a3ee 100644 +--- a/Lib/zipfile.py ++++ b/Lib/zipfile.py +@@ -9,6 +9,7 @@ import io + import itertools + import os + import posixpath ++import re + import shutil + import stat + import struct +@@ -2192,7 +2193,7 @@ def _parents(path): + def _ancestry(path): + """ + Given a path with elements separated by +- posixpath.sep, generate all elements of that path ++ posixpath.sep, generate all elements of that path. + + >>> list(_ancestry('b/d')) + 'b/d', 'b' +@@ -2204,9 +2205,14 @@ def _ancestry(path): + 'b' + >>> list(_ancestry('')) + ++ ++ Multiple separators are treated like a single. ++ ++ >>> list(_ancestry('//b//d///f//')) ++ '//b//d///f', '//b//d', '//b' + """ + path = path.rstrip(posixpath.sep) +- while path and path != posixpath.sep: ++ while path.rstrip(posixpath.sep): + yield path + path, tail = posixpath.split(path) + +diff --git a/Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst b/Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst +new file mode 100644 +index 0000000..ee9fde6 +--- /dev/null ++++ b/Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst +@@ -0,0 +1,3 @@ ++Applied a more surgical fix for malformed payloads in :class:`zipfile.Path` ++causing infinite loops (gh-122905) without breaking contents using ++legitimate characters. +-- +2.43.0 +
View file
_service:tar_scm:backport-gh-121650-Encode-newlines-in-headers-and-verify-head.patch
Deleted
@@ -1,348 +0,0 @@ -From 67067f7cca97cc980d31c5769f621b58ef876b7e Mon Sep 17 00:00:00 2001 -From: xinsheng <xinsheng3@huawei.com> -Date: Tue, 3 Sep 2024 21:17:26 +0800 -Subject: PATCH gh-121650: Encode newlines in headers, and verify headers are - sound (GH-122233) - ---- - Doc/library/email.errors.rst | 6 ++ - Doc/library/email.policy.rst | 18 ++++++ - Doc/whatsnew/3.11.rst | 13 ++++ - Lib/email/_header_value_parser.py | 12 +++- - Lib/email/_policybase.py | 8 +++ - Lib/email/errors.py | 4 ++ - Lib/email/generator.py | 13 +++- - Lib/test/test_email/test_generator.py | 62 +++++++++++++++++++ - Lib/test/test_email/test_policy.py | 26 ++++++++ - ...-07-27-16-10-41.gh-issue-121650.nf6oc9.rst | 5 ++ - 10 files changed, 163 insertions(+), 4 deletions(-) - create mode 100644 Misc/NEWS.d/next/Library/2024-07-27-16-10-41.gh-issue-121650.nf6oc9.rst - -diff --git a/Doc/library/email.errors.rst b/Doc/library/email.errors.rst -index 194a986..f737f02 100644 ---- a/Doc/library/email.errors.rst -+++ b/Doc/library/email.errors.rst -@@ -59,6 +59,12 @@ The following exception classes are defined in the :mod:`email.errors` module: - :class:`~email.mime.image.MIMEImage`). - - -+.. exception:: HeaderWriteError() -+ -+ Raised when an error occurs when the :mod:`~email.generator` outputs -+ headers. -+ -+ - Here is the list of the defects that the :class:`~email.parser.FeedParser` - can find while parsing messages. Note that the defects are added to the message - where the problem was found, so for example, if a message nested inside a -diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst -index bf53b95..79b1786 100644 ---- a/Doc/library/email.policy.rst -+++ b/Doc/library/email.policy.rst -@@ -229,6 +229,24 @@ added matters. To illustrate:: - - .. versionadded:: 3.6 - -+ -+ .. attribute:: verify_generated_headers -+ -+ If ``True`` (the default), the generator will raise -+ :exc:`~email.errors.HeaderWriteError` instead of writing a header -+ that is improperly folded or delimited, such that it would -+ be parsed as multiple headers or joined with adjacent data. -+ Such headers can be generated by custom header classes or bugs -+ in the ``email`` module. -+ -+ As it's a security feature, this defaults to ``True`` even in the -+ :class:`~email.policy.Compat32` policy. -+ For backwards compatible, but unsafe, behavior, it must be set to -+ ``False`` explicitly. -+ -+ .. versionadded:: 3.11.10 -+ -+ - The following :class:`Policy` method is intended to be called by code using - the email library to create policy instances with custom settings: - -diff --git a/Doc/whatsnew/3.11.rst b/Doc/whatsnew/3.11.rst -index c47c895..d46df98 100644 ---- a/Doc/whatsnew/3.11.rst -+++ b/Doc/whatsnew/3.11.rst -@@ -2727,6 +2727,7 @@ OpenSSL - - .. _libb2: https://www.blake2.net/ - -+ - Notable changes in 3.11.10 - ========================== - -@@ -2735,3 +2736,15 @@ ipaddress - - * Fixed ``is_global`` and ``is_private`` behavior in ``IPv4Address``, - ``IPv6Address``, ``IPv4Network`` and ``IPv6Network``. -+ -+email -+----- -+ -+* Headers with embedded newlines are now quoted on output. -+ -+ The :mod:`~email.generator` will now refuse to serialize (write) headers -+ that are improperly folded or delimited, such that they would be parsed as -+ multiple headers or joined with adjacent data. -+ If you need to turn this safety feature off, -+ set :attr:`~email.policy.Policy.verify_generated_headers`. -+ (Contributed by Bas Bloemsaat and Petr Viktorin in :gh:`121650`.) -\ No newline at end of file -diff --git a/Lib/email/_header_value_parser.py b/Lib/email/_header_value_parser.py -index e637e6d..e1b99d5 100644 ---- a/Lib/email/_header_value_parser.py -+++ b/Lib/email/_header_value_parser.py -@@ -92,6 +92,8 @@ TOKEN_ENDS = TSPECIALS | WSP - ASPECIALS = TSPECIALS | set("*'%") - ATTRIBUTE_ENDS = ASPECIALS | WSP - EXTENDED_ATTRIBUTE_ENDS = ATTRIBUTE_ENDS - set('%') -+NLSET = {'\n', '\r'} -+SPECIALSNL = SPECIALS | NLSET - - def quote_string(value): - return '"'+str(value).replace('\\', '\\\\').replace('"', r'\"')+'"' -@@ -2778,9 +2780,13 @@ def _refold_parse_tree(parse_tree, *, policy): - wrap_as_ew_blocked -= 1 - continue - tstr = str(part) -- if part.token_type == 'ptext' and set(tstr) & SPECIALS: -- # Encode if tstr contains special characters. -- want_encoding = True -+ if not want_encoding: -+ if part.token_type == 'ptext': -+ # Encode if tstr contains special characters. -+ want_encoding = not SPECIALSNL.isdisjoint(tstr) -+ else: -+ # Encode if tstr contains newlines. -+ want_encoding = not NLSET.isdisjoint(tstr) - try: - tstr.encode(encoding) - charset = encoding -diff --git a/Lib/email/_policybase.py b/Lib/email/_policybase.py -index c9cbadd..d1f4821 100644 ---- a/Lib/email/_policybase.py -+++ b/Lib/email/_policybase.py -@@ -157,6 +157,13 @@ class Policy(_PolicyBase, metaclass=abc.ABCMeta): - message_factory -- the class to use to create new message objects. - If the value is None, the default is Message. - -+ verify_generated_headers -+ -- if true, the generator verifies that each header -+ they are properly folded, so that a parser won't -+ treat it as multiple headers, start-of-body, or -+ part of another header. -+ This is a check against custom Header & fold() -+ implementations. - """ - - raise_on_defect = False -@@ -165,6 +172,7 @@ class Policy(_PolicyBase, metaclass=abc.ABCMeta): - max_line_length = 78 - mangle_from_ = False - message_factory = None -+ verify_generated_headers = True - - def handle_defect(self, obj, defect): - """Based on policy, either raise defect or call register_defect. -diff --git a/Lib/email/errors.py b/Lib/email/errors.py -index 3ad0056..02aa5ec 100644 ---- a/Lib/email/errors.py -+++ b/Lib/email/errors.py -@@ -29,6 +29,10 @@ class CharsetError(MessageError): - """An illegal charset was given.""" - - -+class HeaderWriteError(MessageError): -+ """Error while writing headers.""" -+ -+ - # These are parsing defects which the parser was able to work around. - class MessageDefect(ValueError): - """Base class for a message defect.""" -diff --git a/Lib/email/generator.py b/Lib/email/generator.py -index b8c1091..9cdd95f 100644 ---- a/Lib/email/generator.py -+++ b/Lib/email/generator.py -@@ -14,12 +14,14 @@ import random - from copy import deepcopy - from io import StringIO, BytesIO - from email.utils import _has_surrogates -+from email.errors import HeaderWriteError - - UNDERSCORE = '_' - NL = '\n' # XXX: no longer used by the code below. - - NLCRE = re.compile(r'\r\n|\r|\n') - fcre = re.compile(r'^From ', re.MULTILINE) -+NEWLINE_WITHOUT_FWSP = re.compile(r'\r\n^ \t|\r^ \n\t|\n^ \t') - - - class Generator: -@@ -222,7 +224,16 @@ class Generator: - - def _write_headers(self, msg): - for h, v in msg.raw_items(): -- self.write(self.policy.fold(h, v)) -+ folded = self.policy.fold(h, v) -+ if self.policy.verify_generated_headers: -+ linesep = self.policy.linesep -+ if not folded.endswith(self.policy.linesep): -+ raise HeaderWriteError( -+ f'folded header does not end with {linesep!r}: {folded!r}') -+ if NEWLINE_WITHOUT_FWSP.search(folded.removesuffix(linesep)): -+ raise HeaderWriteError( -+ f'folded header contains newline: {folded!r}')
View file
_service:tar_scm:backport-gh-123067-Fix-quadratic-complexity-in-parsing-quoted.patch
Deleted
@@ -1,131 +0,0 @@ -From cc22eb02afab967715da592536fc5eeab7d7c765 Mon Sep 17 00:00:00 2001 -From: xinsheng <xinsheng3@huawei.com> -Date: Tue, 3 Sep 2024 14:58:02 +0800 -Subject: PATCH gh-123067: Fix quadratic complexity in parsing ^Cuoted cookie - values with backslashes - ---- - Lib/http/cookies.py | 34 ++++------------- - Lib/test/test_http_cookies.py | 38 +++++++++++++++++++ - ...-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst | 1 + - 3 files changed, 47 insertions(+), 26 deletions(-) - create mode 100644 Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst - -diff --git a/Lib/http/cookies.py b/Lib/http/cookies.py -index 35ac2dc..2c1f021 100644 ---- a/Lib/http/cookies.py -+++ b/Lib/http/cookies.py -@@ -184,8 +184,13 @@ def _quote(str): - return '"' + str.translate(_Translator) + '"' - - --_OctalPatt = re.compile(r"\\0-30-70-7") --_QuotePatt = re.compile(r"\\.") -+_unquote_sub = re.compile(r'\\(?:(0-30-70-7)|(.))').sub -+ -+def _unquote_replace(m): -+ if m1: -+ return chr(int(m1, 8)) -+ else: -+ return m2 - - def _unquote(str): - # If there aren't any doublequotes, -@@ -205,30 +210,7 @@ def _unquote(str): - # \012 --> \n - # \" --> " - # -- i = 0 -- n = len(str) -- res = -- while 0 <= i < n: -- o_match = _OctalPatt.search(str, i) -- q_match = _QuotePatt.search(str, i) -- if not o_match and not q_match: # Neither matched -- res.append(stri:) -- break -- # else: -- j = k = -1 -- if o_match: -- j = o_match.start(0) -- if q_match: -- k = q_match.start(0) -- if q_match and (not o_match or k < j): # QuotePatt matched -- res.append(stri:k) -- res.append(strk+1) -- i = k + 2 -- else: # OctalPatt matched -- res.append(stri:j) -- res.append(chr(int(strj+1:j+4, 8))) -- i = j + 4 -- return _nulljoin(res) -+ return _unquote_sub(_unquote_replace, str) - - # The _getdate() routine is used to set the expiration time in the cookie's HTTP - # header. By default, _getdate() returns the current time in the appropriate -diff --git a/Lib/test/test_http_cookies.py b/Lib/test/test_http_cookies.py -index 925c869..8879902 100644 ---- a/Lib/test/test_http_cookies.py -+++ b/Lib/test/test_http_cookies.py -@@ -5,6 +5,7 @@ import unittest - import doctest - from http import cookies - import pickle -+from test import support - - - class CookieTests(unittest.TestCase): -@@ -58,6 +59,43 @@ class CookieTests(unittest.TestCase): - for k, v in sorted(case'dict'.items()): - self.assertEqual(Ck.value, v) - -+ def test_unquote(self): -+ cases = -+ (r'a="b=\""', 'b="'), -+ (r'a="b=\\"', 'b=\\'), -+ (r'a="b=\="', 'b=='), -+ (r'a="b=\n"', 'b=n'), -+ (r'a="b=\042"', 'b="'), -+ (r'a="b=\134"', 'b=\\'), -+ (r'a="b=\377"', 'b=\xff'), -+ (r'a="b=\400"', 'b=400'), -+ (r'a="b=\42"', 'b=42'), -+ (r'a="b=\\042"', 'b=\\042'), -+ (r'a="b=\\134"', 'b=\\134'), -+ (r'a="b=\\\""', 'b=\\"'), -+ (r'a="b=\\\042"', 'b=\\"'), -+ (r'a="b=\134\""', 'b=\\"'), -+ (r'a="b=\134\042"', 'b=\\"'), -+ -+ for encoded, decoded in cases: -+ with self.subTest(encoded): -+ C = cookies.SimpleCookie() -+ C.load(encoded) -+ self.assertEqual(C'a'.value, decoded) -+ -+ @support.requires_resource('cpu') -+ def test_unquote_large(self): -+ n = 10**6 -+ for encoded in r'\\', r'\134': -+ with self.subTest(encoded): -+ data = 'a="b=' + encoded*n + ';"' -+ C = cookies.SimpleCookie() -+ C.load(data) -+ value = C'a'.value -+ self.assertEqual(value:3, 'b=\\') -+ self.assertEqual(value-2:, '\\;') -+ self.assertEqual(len(value), n + 3) -+ - def test_load(self): - C = cookies.SimpleCookie() - C.load('Customer="WILE_E_COYOTE"; Version=1; Path=/acme') -diff --git a/Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst b/Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst -new file mode 100644 -index 0000000..6a23456 ---- /dev/null -+++ b/Misc/NEWS.d/next/Library/2024-08-16-19-13-21.gh-issue-123067.Nx9O4R.rst -@@ -0,0 +1 @@ -+Fix quadratic complexity in parsing ``"``-quoted cookie values with backslashes by :mod:`http.cookies`. --- -2.43.0 -
View file
_service:tar_scm:backport-gh-123270-Replaced-SanitizedNames-with-a-more-surgic.patch
Deleted
@@ -1,150 +0,0 @@ -From 148c957b6f1fe22ed5465f4b05adb87db361fb0f Mon Sep 17 00:00:00 2001 -From: xinsheng <xinsheng3@huawei.com> -Date: Wed, 4 Sep 2024 19:21:41 +0800 -Subject: PATCH gh-123270: Replaced SanitizedNames with a more surgical fix. - (GH-123354) - ---- - Lib/test/test_zipfile.py | 77 +++++++++++++++++++ - Lib/zipfile.py | 10 ++- - ...-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst | 3 + - 3 files changed, 88 insertions(+), 2 deletions(-) - create mode 100644 Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst - -diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py -index c8e0159..669e494 100644 ---- a/Lib/test/test_zipfile.py -+++ b/Lib/test/test_zipfile.py -@@ -3512,6 +3512,83 @@ with zipfile.ZipFile(io.BytesIO(), "w") as zf: - zipfile.Path(zf) - zf.extractall(source_path.parent) - -+ def test_malformed_paths(self): -+ """ -+ Path should handle malformed paths gracefully. -+ -+ Paths with leading slashes are not visible. -+ -+ Paths with dots are treated like regular files. -+ """ -+ data = io.BytesIO() -+ zf = zipfile.ZipFile(data, "w") -+ zf.writestr("/one-slash.txt", b"content") -+ zf.writestr("//two-slash.txt", b"content") -+ zf.writestr("../parent.txt", b"content") -+ zf.filename = '' -+ root = zipfile.Path(zf) -+ assert list(map(str, root.iterdir())) == '../' -+ assert root.joinpath('..').joinpath('parent.txt').read_bytes() == b'content' -+ -+ def test_unsupported_names(self): -+ """ -+ Path segments with special characters are readable. -+ -+ On some platforms or file systems, characters like -+ ``:`` and ``?`` are not allowed, but they are valid -+ in the zip file. -+ """ -+ data = io.BytesIO() -+ zf = zipfile.ZipFile(data, "w") -+ zf.writestr("path?", b"content") -+ zf.writestr("V: NMS.flac", b"fLaC...") -+ zf.filename = '' -+ root = zipfile.Path(zf) -+ contents = root.iterdir() -+ assert next(contents).name == 'path?' -+ assert next(contents).name == 'V: NMS.flac' -+ assert root.joinpath('V: NMS.flac').read_bytes() == b"fLaC..." -+ -+ def test_backslash_not_separator(self): -+ """ -+ In a zip file, backslashes are not separators. -+ """ -+ data = io.BytesIO() -+ zf = zipfile.ZipFile(data, "w") -+ zf.writestr(DirtyZipInfo.for_name("foo\\bar", zf), b"content") -+ zf.filename = '' -+ root = zipfile.Path(zf) -+ (first,) = root.iterdir() -+ assert not first.is_dir() -+ assert first.name == 'foo\\bar' -+ -+ -+class DirtyZipInfo(zipfile.ZipInfo): -+ """ -+ Bypass name sanitization. -+ """ -+ -+ def __init__(self, filename, *args, **kwargs): -+ super().__init__(filename, *args, **kwargs) -+ self.filename = filename -+ -+ @classmethod -+ def for_name(cls, name, archive): -+ """ -+ Construct the same way that ZipFile.writestr does. -+ -+ TODO: extract this functionality and re-use -+ """ -+ self = cls(filename=name, date_time=time.localtime(time.time()):6) -+ self.compress_type = archive.compression -+ self.compress_level = archive.compresslevel -+ if self.filename.endswith('/'): # pragma: no cover -+ self.external_attr = 0o40775 << 16 # drwxrwxr-x -+ self.external_attr |= 0x10 # MS-DOS directory flag -+ else: -+ self.external_attr = 0o600 << 16 # ?rw------- -+ return self -+ - - class EncodedMetadataTests(unittest.TestCase): - file_names = '\u4e00', '\u4e8c', '\u4e09' # Han 'one', 'two', 'three' -diff --git a/Lib/zipfile.py b/Lib/zipfile.py -index 6189db5..622a3ee 100644 ---- a/Lib/zipfile.py -+++ b/Lib/zipfile.py -@@ -9,6 +9,7 @@ import io - import itertools - import os - import posixpath -+import re - import shutil - import stat - import struct -@@ -2192,7 +2193,7 @@ def _parents(path): - def _ancestry(path): - """ - Given a path with elements separated by -- posixpath.sep, generate all elements of that path -+ posixpath.sep, generate all elements of that path. - - >>> list(_ancestry('b/d')) - 'b/d', 'b' -@@ -2204,9 +2205,14 @@ def _ancestry(path): - 'b' - >>> list(_ancestry('')) - -+ -+ Multiple separators are treated like a single. -+ -+ >>> list(_ancestry('//b//d///f//')) -+ '//b//d///f', '//b//d', '//b' - """ - path = path.rstrip(posixpath.sep) -- while path and path != posixpath.sep: -+ while path.rstrip(posixpath.sep): - yield path - path, tail = posixpath.split(path) - -diff --git a/Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst b/Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst -new file mode 100644 -index 0000000..ee9fde6 ---- /dev/null -+++ b/Misc/NEWS.d/next/Library/2024-08-26-13-45-20.gh-issue-123270.gXHvNJ.rst -@@ -0,0 +1,3 @@ -+Applied a more surgical fix for malformed payloads in :class:`zipfile.Path` -+causing infinite loops (gh-122905) without breaking contents using -+legitimate characters. --- -2.43.0 -
Locations
Projects
Search
Status Monitor
Help
Open Build Service
OBS Manuals
API Documentation
OBS Portal
Reporting a Bug
Contact
Mailing List
Forums
Chat (IRC)
Twitter
Open Build Service (OBS)
is an
openSUSE project
.
浙ICP备2022010568号-2