Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support reading remote files / directories / datasources #2171

Open
IvanGao01 opened this issue Jun 7, 2024 · 0 comments
Open

Support reading remote files / directories / datasources #2171

IvanGao01 opened this issue Jun 7, 2024 · 0 comments
Assignees
Labels
type/feature-request New feature or request

Comments

@IvanGao01
Copy link
Member

IvanGao01 commented Jun 7, 2024

Is your feature request related to a problem? Please describe.

Supports reading remote directories or files, such as https, S3, etc.

Describe the solution you'd like
Example 1:

select count(*) from 'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet'
+----------+
| COUNT(*) |
+----------+
| 1000000  |
+----------+
1 row in set. Query took 0.595 seconds.

Example 2:

CREATE EXTERNAL TABLE hits
STORED AS PARQUET
LOCATION 'https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_1.parquet';
select count(*) from hits;
+----------+
| COUNT(*) |
+----------+
| 1000000  |
+----------+

Additional context

Reference: https://datafusion.apache.org/user-guide/cli/datasources.html#

In addition to this, the ability to read other data sources and provide the ability to Join with other data sources would greatly enhance the ease of use of the product, such as using time series tables in CnosDB to Join with dimension tables in PostgreSQL or Mysql.

@IvanGao01 IvanGao01 added the type/feature-request New feature or request label Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature-request New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants