Skip to content

Commit 310455c

Browse files
miss-islingtonlysnikolaouStanFromIreland
authored
[3.14] gh-142518: Move thread safety sections into a new page (GH-144716) (#145223)
- Create a new page for thread safety notes for built-in types - Move thread safety notes for `list` into the new page - Move thread safety notes for `dict` into the new page --------- (cherry picked from commit 017ccd3) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
1 parent 0701ce6 commit 310455c

File tree

3 files changed

+269
-241
lines changed

3 files changed

+269
-241
lines changed

Doc/library/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ the `Python Package Index <https://pypi.org>`_.
4343
constants.rst
4444
stdtypes.rst
4545
exceptions.rst
46+
threadsafety.rst
4647

4748
text.rst
4849
binary.rst

Doc/library/stdtypes.rst

Lines changed: 6 additions & 241 deletions
Original file line numberDiff line numberDiff line change
@@ -1449,111 +1449,10 @@ application).
14491449
list appear empty for the duration, and raises :exc:`ValueError` if it can
14501450
detect that the list has been mutated during a sort.
14511451

1452-
.. _thread-safety-list:
1453-
1454-
.. rubric:: Thread safety for list objects
1455-
1456-
Reading a single element from a :class:`list` is
1457-
:term:`atomic <atomic operation>`:
1458-
1459-
.. code-block::
1460-
:class: green
1461-
1462-
lst[i] # list.__getitem__
1463-
1464-
The following methods traverse the list and use :term:`atomic <atomic operation>`
1465-
reads of each item to perform their function. That means that they may
1466-
return results affected by concurrent modifications:
1467-
1468-
.. code-block::
1469-
:class: maybe
1470-
1471-
item in lst
1472-
lst.index(item)
1473-
lst.count(item)
1474-
1475-
All of the above operations avoid acquiring :term:`per-object locks
1476-
<per-object lock>`. They do not block concurrent modifications. Other
1477-
operations that hold a lock will not block these from observing intermediate
1478-
states.
1479-
1480-
All other operations from here on block using the :term:`per-object lock`.
1481-
1482-
Writing a single item via ``lst[i] = x`` is safe to call from multiple
1483-
threads and will not corrupt the list.
1484-
1485-
The following operations return new objects and appear
1486-
:term:`atomic <atomic operation>` to other threads:
1487-
1488-
.. code-block::
1489-
:class: good
1490-
1491-
lst1 + lst2 # concatenates two lists into a new list
1492-
x * lst # repeats lst x times into a new list
1493-
lst.copy() # returns a shallow copy of the list
1494-
1495-
The following methods that only operate on a single element with no shifting
1496-
required are :term:`atomic <atomic operation>`:
1497-
1498-
.. code-block::
1499-
:class: good
1500-
1501-
lst.append(x) # append to the end of the list, no shifting required
1502-
lst.pop() # pop element from the end of the list, no shifting required
1503-
1504-
The :meth:`~list.clear` method is also :term:`atomic <atomic operation>`.
1505-
Other threads cannot observe elements being removed.
1506-
1507-
The :meth:`~list.sort` method is not :term:`atomic <atomic operation>`.
1508-
Other threads cannot observe intermediate states during sorting, but the
1509-
list appears empty for the duration of the sort.
1510-
1511-
The following operations may allow :term:`lock-free` operations to observe
1512-
intermediate states since they modify multiple elements in place:
1513-
1514-
.. code-block::
1515-
:class: maybe
1516-
1517-
lst.insert(idx, item) # shifts elements
1518-
lst.pop(idx) # idx not at the end of the list, shifts elements
1519-
lst *= x # copies elements in place
1520-
1521-
The :meth:`~list.remove` method may allow concurrent modifications since
1522-
element comparison may execute arbitrary Python code (via
1523-
:meth:`~object.__eq__`).
1524-
1525-
:meth:`~list.extend` is safe to call from multiple threads. However, its
1526-
guarantees depend on the iterable passed to it. If it is a :class:`list`, a
1527-
:class:`tuple`, a :class:`set`, a :class:`frozenset`, a :class:`dict` or a
1528-
:ref:`dictionary view object <dict-views>` (but not their subclasses), the
1529-
``extend`` operation is safe from concurrent modifications to the iterable.
1530-
Otherwise, an iterator is created which can be concurrently modified by
1531-
another thread. The same applies to inplace concatenation of a list with
1532-
other iterables when using ``lst += iterable``.
1533-
1534-
Similarly, assigning to a list slice with ``lst[i:j] = iterable`` is safe
1535-
to call from multiple threads, but ``iterable`` is only locked when it is
1536-
also a :class:`list` (but not its subclasses).
1537-
1538-
Operations that involve multiple accesses, as well as iteration, are never
1539-
atomic. For example:
1540-
1541-
.. code-block::
1542-
:class: bad
1543-
1544-
# NOT atomic: read-modify-write
1545-
lst[i] = lst[i] + 1
1546-
1547-
# NOT atomic: check-then-act
1548-
if lst:
1549-
item = lst.pop()
1550-
1551-
# NOT thread-safe: iteration while modifying
1552-
for item in lst:
1553-
process(item) # another thread may modify lst
1452+
.. seealso::
15541453

1555-
Consider external synchronization when sharing :class:`list` instances
1556-
across threads. See :ref:`freethreading-python-howto` for more information.
1454+
For detailed information on thread-safety guarantees for :class:`list`
1455+
objects, see :ref:`thread-safety-list`.
15571456

15581457

15591458
.. _typesseq-tuple:
@@ -5569,144 +5468,10 @@ can be used interchangeably to index the same dictionary entry.
55695468
of a :class:`dict`.
55705469

55715470

5572-
.. _thread-safety-dict:
5573-
5574-
.. rubric:: Thread safety for dict objects
5575-
5576-
Creating a dictionary with the :class:`dict` constructor is atomic when the
5577-
argument to it is a :class:`dict` or a :class:`tuple`. When using the
5578-
:meth:`dict.fromkeys` method, dictionary creation is atomic when the
5579-
argument is a :class:`dict`, :class:`tuple`, :class:`set` or
5580-
:class:`frozenset`.
5581-
5582-
The following operations and functions are :term:`lock-free` and
5583-
:term:`atomic <atomic operation>`.
5584-
5585-
.. code-block::
5586-
:class: good
5587-
5588-
d[key] # dict.__getitem__
5589-
d.get(key) # dict.get
5590-
key in d # dict.__contains__
5591-
len(d) # dict.__len__
5592-
5593-
All other operations from here on hold the :term:`per-object lock`.
5594-
5595-
Writing or removing a single item is safe to call from multiple threads
5596-
and will not corrupt the dictionary:
5597-
5598-
.. code-block::
5599-
:class: good
5600-
5601-
d[key] = value # write
5602-
del d[key] # delete
5603-
d.pop(key) # remove and return
5604-
d.popitem() # remove and return last item
5605-
d.setdefault(key, v) # insert if missing
5606-
5607-
These operations may compare keys using :meth:`~object.__eq__`, which can
5608-
execute arbitrary Python code. During such comparisons, the dictionary may
5609-
be modified by another thread. For built-in types like :class:`str`,
5610-
:class:`int`, and :class:`float`, that implement :meth:`~object.__eq__` in C,
5611-
the underlying lock is not released during comparisons and this is not a
5612-
concern.
5613-
5614-
The following operations return new objects and hold the :term:`per-object lock`
5615-
for the duration of the operation:
5616-
5617-
.. code-block::
5618-
:class: good
5619-
5620-
d.copy() # returns a shallow copy of the dictionary
5621-
d | other # merges two dicts into a new dict
5622-
d.keys() # returns a new dict_keys view object
5623-
d.values() # returns a new dict_values view object
5624-
d.items() # returns a new dict_items view object
5625-
5626-
The :meth:`~dict.clear` method holds the lock for its duration. Other
5627-
threads cannot observe elements being removed.
5628-
5629-
The following operations lock both dictionaries. For :meth:`~dict.update`
5630-
and ``|=``, this applies only when the other operand is a :class:`dict`
5631-
that uses the standard dict iterator (but not subclasses that override
5632-
iteration). For equality comparison, this applies to :class:`dict` and
5633-
its subclasses:
5634-
5635-
.. code-block::
5636-
:class: good
5637-
5638-
d.update(other_dict) # both locked when other_dict is a dict
5639-
d |= other_dict # both locked when other_dict is a dict
5640-
d == other_dict # both locked for dict and subclasses
5641-
5642-
All comparison operations also compare values using :meth:`~object.__eq__`,
5643-
so for non-built-in types the lock may be released during comparison.
5644-
5645-
:meth:`~dict.fromkeys` locks both the new dictionary and the iterable
5646-
when the iterable is exactly a :class:`dict`, :class:`set`, or
5647-
:class:`frozenset` (not subclasses):
5648-
5649-
.. code-block::
5650-
:class: good
5651-
5652-
dict.fromkeys(a_dict) # locks both
5653-
dict.fromkeys(a_set) # locks both
5654-
dict.fromkeys(a_frozenset) # locks both
5655-
5656-
When updating from a non-dict iterable, only the target dictionary is
5657-
locked. The iterable may be concurrently modified by another thread:
5658-
5659-
.. code-block::
5660-
:class: maybe
5661-
5662-
d.update(iterable) # iterable is not a dict: only d locked
5663-
d |= iterable # iterable is not a dict: only d locked
5664-
dict.fromkeys(iterable) # iterable is not a dict/set/frozenset: only result locked
5665-
5666-
Operations that involve multiple accesses, as well as iteration, are never
5667-
atomic:
5668-
5669-
.. code-block::
5670-
:class: bad
5671-
5672-
# NOT atomic: read-modify-write
5673-
d[key] = d[key] + 1
5674-
5675-
# NOT atomic: check-then-act (TOCTOU)
5676-
if key in d:
5677-
del d[key]
5678-
5679-
# NOT thread-safe: iteration while modifying
5680-
for key, value in d.items():
5681-
process(key) # another thread may modify d
5682-
5683-
To avoid time-of-check to time-of-use (TOCTOU) issues, use atomic
5684-
operations or handle exceptions:
5685-
5686-
.. code-block::
5687-
:class: good
5688-
5689-
# Use pop() with default instead of check-then-delete
5690-
d.pop(key, None)
5691-
5692-
# Or handle the exception
5693-
try:
5694-
del d[key]
5695-
except KeyError:
5696-
pass
5697-
5698-
To safely iterate over a dictionary that may be modified by another
5699-
thread, iterate over a copy:
5700-
5701-
.. code-block::
5702-
:class: good
5703-
5704-
# Make a copy to iterate safely
5705-
for key, value in d.copy().items():
5706-
process(key)
5471+
.. seealso::
57075472

5708-
Consider external synchronization when sharing :class:`dict` instances
5709-
across threads. See :ref:`freethreading-python-howto` for more information.
5473+
For detailed information on thread-safety guarantees for :class:`dict`
5474+
objects, see :ref:`thread-safety-dict`.
57105475

57115476

57125477
.. _dict-views:

0 commit comments

Comments
 (0)