Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.5.0, 4.0.0
Description
https://github.com/apache/spark/actions/runs/6914662405/job/18812759697
====================================================================== ERROR [0.686s]: test_apply_batch_with_type (pyspark.pandas.tests.computation.test_apply_func.FrameApplyFunctionTests.test_apply_batch_with_type) ---------------------------------------------------------------------- Traceback (most recent call last): File "/__w/spark/spark/python/pyspark/pandas/tests/computation/test_apply_func.py", line 248, in test_apply_batch_with_type def identify3(x) -> ps.DataFrame[float, [int, List[int]]]: ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ File "/__w/spark/spark/python/pyspark/pandas/frame.py", line 13540, in __class_getitem__ return create_tuple_for_frame_type(params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/__w/spark/spark/python/pyspark/pandas/typedef/typehints.py", line 721, in create_tuple_for_frame_type return Tuple[_to_type_holders(params)] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/__w/spark/spark/python/pyspark/pandas/typedef/typehints.py", line 766, in _to_type_holders data_types = _new_type_holders(data_types, NameTypeHolder) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/__w/spark/spark/python/pyspark/pandas/typedef/typehints.py", line 832, in _new_type_holders raise TypeError( TypeError: Type hints should be specified as one of: - DataFrame[type, type, ...] - DataFrame[name: type, name: type, ...] - DataFrame[dtypes instance] - DataFrame[zip(names, types)] - DataFrame[index_type, [type, ...]] - DataFrame[(index_name, index_type), [(name, type), ...]] - DataFrame[dtype instance, dtypes instance] - DataFrame[(index_name, index_type), zip(names, types)] - DataFrame[[index_type, ...], [type, ...]] - DataFrame[[(index_name, index_type), ...], [(name, type), ...]] - DataFrame[dtypes instance, dtypes instance] - DataFrame[zip(index_names, index_types), zip(names, types)] However, got (<class 'int'>, typing.List[int]). ---------------------------------------------------------------------- Ran 10 tests in 34.327s FAILED (errors=1)
Attachments
Issue Links
- is related to
-
SPARK-45989 Fix `pyspark.pandas.tests.connect.computation.test_parity_apply_func` in Python 3.11
- Resolved
- links to