PERF: Avoid intermediate `ndarray[object]` array when constructing a pyarrow-backed string data from untyped data

[mroeschke]

In [1]: import pandas as pd

In [2]: pd.Series(["a"]).array
> /pandas/core/construction.py(695)sanitize_array()
-> subarr = maybe_convert_platform(data)
(Pdb) data
['a']
(Pdb) n
> /pandas/core/construction.py(696)sanitize_array()
-> if subarr.dtype == object:
(Pdb) subarr
array(['a'], dtype=object)
(Pdb) c
Out[5]: 
<ArrowStringArray>
['a']
Length: 1, dtype: str

In [3]: pd.Series(["a"], dtype=pd.StringDtype("pyarrow"))
Out[3]: 
0    a
dtype: string

maybe_convert_platform calls construct_1d_object_array_from_listlike to convert the list to ndarray[object] for further inference. I wonder if we can perform type inference without converting to object such that for strings we can pass the list directly to ArrowStringArray._from_sequence_of_strings

[sanrishi]

I've been researching on this and wanted to run a proposed approach by you before finalizing a PR.

Currently, in sanitize_array, when a raw list of strings is passed, it falls through to maybe_convert_platform(data). This eventually forces the sequence into a NumPy ndarray[object] via construct_1d_object_array_from_listlike and lib.maybe_convert_objects before PyArrow type inference even occurs.

We can intercept the raw sequence in sanitize_array before it hits maybe_convert_platform.

1. Check if using_string_dtype() is active and the target storage is pyarrow.

2. Do a quick check on the list data (ignoring NAs). If it contains strings, bypass NumPy entirely.

3. Pass the sequence directly to PyArrow using pa.array(data, type=pa.large_string(), from_pandas=True).

4. Wrap the resulting PyArrow array in ArrowStringArray.

I've locally prototyped this, and it successfully routes around the intermediate ndarray[object] allocation. The from_pandas=True flag in PyArrow correctly handles missing values (None, np.nan, pd.NA) as nulls.

Does this architectural approach align with the long-term goals for infer_string defaults? Let me know if you have any feedback or if I should proceed with polishing this up for a PR!

Best,
Sanchit