Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28841

Spark cannot read a relative path containing ":"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.4.3
    • None
    • SQL

    Description

      Reproducer:

      spark.read.parquet("test:test")

      Error:

      java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: test:test

      This is actually a Hadoop issue since the error is thrown from "new Path("test:test")". I'm creating this ticket to see if we can work around this issue in Spark.

      Attachments

        Issue Links

          is caused by

          Bug - A problem which impairs or prevents the functions of the product. HDFS-14762 "Path(Path/String parent, String child)" will fail when "child" contains ":"

          • Major - Major loss of function.
          • Reopened

          Activity

            People

              Unassigned Unassigned
              zsxwing Shixiong Zhu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: