Open
Description
Describe the bug
Hi
I was trying to create external table from AWS S3 using examples from Datafusion CLI docs and SQL DDL docs, see To Reproduce section below. But looks like AWS-specific options for external tables doesn't work with SQL API in the same way as it described in CLI docs
Exception: DataFusion error: Configuration("Could not find config namespace \"aws\"")
I can successfully register and read the same parquet with the same credentials via register_object_store + register_parquet functions, so there's no problem with the data or S3 connection
To Reproduce
import datafusion
ctx = datafusion.SessionContext()
raw_data_df = ctx.sql(f"""
create external table raw_data
stored as parquet
options
(
'aws.access_key_id' '***',
'aws.secret_access_key' '***',
'aws.region' 'eu-central-1'
)
location 's3://some-bucket/some-folder/'
""")
raw_data_df.show()
Expected behavior
create external table
command works with aws.xxx
options
Additional context
v 42.0.0