Commit Graph

96 Commits

Author SHA1 Message Date
Joe
ff3617767d
feat: support postgresql (#286) 2023-08-29 09:06:11 -07:00
Charles Swanberg
01c5422206
Cloud support deltalake (#280)
* fix: enable cloud support for deltalake

* fix clippy issues

* formatting fixes
2023-06-20 00:43:35 -07:00
QP Hou
df39a13d3f
upgrade datafusion and arrow (#279)
* update arrow, datafusion, delta, convergence, object_store, sqlparser

* fix breaking code

* update to datafusion 23

* update connectorx fork

---------

Co-authored-by: Chitral Verma <chitralverma@gmail.com>
2023-05-29 23:25:45 -07:00
Jiayu Liu
ae13d3bc9e
support toml as config lang (#277)
* support toml as config lang

* update readme
2023-05-22 22:45:39 -07:00
Akshith Madhur
4cd9360fb4
fixed csv-s3 issue (#268)
Co-authored-by: Elliot Alderson <elliot@Elliots-MacBook-Pro.local>
2023-03-14 23:35:02 -07:00
Qingping Hou
5b28b7c252 support catalog queries 2023-02-06 19:02:20 -08:00
Qingping Hou
c91e9cbe57 apply and enforce rust-fmt 2023-02-06 19:02:20 -08:00
Rich
f9b17888cb
fix url decoding of space (#257)
* Add two test case of space

* decode percent in url

* remove query of space
2023-01-30 23:06:01 -08:00
Ryan Russell
97e6a4cd8d
chore(columnq): readability fixups in json (#256) 2023-01-29 09:43:39 -08:00
QP Hou
c8e7b596ec
ci: add sqlite tests (#253) 2023-01-28 23:55:54 -08:00
QP Hou
2c128c8f7f
ci: fix lints from new rust release (#254) 2023-01-28 16:44:03 -08:00
Rich
2ec39ddba8
support Native Azure Storage (#250)
* support native Azure Storage

* query azure_blogs in CI

* check Azure Storage support in readme

* remove debug print

* Handle url.host_str error
2023-01-23 18:25:56 -08:00
Rich
fffc8558a8
Native GCS support (#248)
* Set object_store minimal version and turn on aws_profile feature

* Add native GCS support with GOOGLE_APPLICATION_CREDENTIALS

* hacky: create /tmp/gcs.json for test

* use GoogleCloudStorageBuilder::from_env

* create temp file inside test for GCS

Co-authored-by: QP Hou <dave2008713@gmail.com>
2023-01-22 07:42:47 +00:00
Rich
e4f00489d1
#240 Set object_store minimal version and turn on aws_profile feature (#247)
* Set object_store minimal version and turn on aws_profile feature

* remove unnecessary with_endpoint after upgrade to object_store 0.5.3
2023-01-21 23:20:11 -08:00
Rich
bd168d92d8
replace rusoto_s3: part 2 (last part) (#245)
* replace rusoto_s3 in columnq::table::delta

* remove rusoto crate
2023-01-18 22:15:12 -08:00
Rich
9db6b418cb
replace s3 table source to unified io:object_store (#244) 2023-01-17 23:29:11 -08:00
Rich
1e3644ec36
#240 support s3 direct with ColumnQObjectStoreProvider (#241)
* Add test case for s3 direct query

* draft: add ColumnQObjectStoreProvider and global Datafusion SessionContext

* fix a few tests of load params change

* fix database tests of load params change

* do not overwrite endpoint if AWS_ENDPOINT_URL is not provided

* ObjectStoreProvider trait change after upgrade from Datafusion 12 to 14

* add comment for difference of test_s3_memory and test_s3_direct

* add unit test for ColumnQObjectStoreProvider

* fix s3 unit test with aws region in CI
2023-01-16 22:03:11 -08:00
QP Hou
3d4da9a810
update datafusion to 14 and arrow to 26 (#242) 2023-01-15 22:47:13 -08:00
Rich
68ee99a924
#18 minio (S3) in CI (#237)
Full end to end S3 test
2023-01-10 22:24:20 -08:00
Rich
eed91fdbbe
#235: upgrade rusoto and force http1 for rustls (#236)
* upgrade rusoto and force http1 for rustls

* add dependency for  openssl_build
2023-01-01 15:33:25 -08:00
Akshith Madhur
c0bff95321
Added Pagination closes #229 (#230)
Co-authored-by: mr-kk-26 <gmail@gmail.com>
2022-11-27 00:33:19 -08:00
Akshith Madhur
026382ecf0
csv use_memory_table option added(resolves issue #224) (#225)
* csv use_memory_table option added

* fixes issue #224
2022-11-21 23:25:36 -08:00
QP Hou
1467739e25
replace dotenv with dotenvy (#216) 2022-11-21 22:15:58 -08:00
Y Togami
beac9cca6a
Fix: support .sqlite, sqlite3 (#223)
* fix: support sqlite extension

* chore: add .sqlite, .sqlite3 test data
2022-11-11 15:47:31 -08:00
Akshith Madhur
7fea7c791a
Fixes issue with limit (#221)
* fixes limit issue

* updated graphql & rest tests
2022-11-07 20:58:35 -08:00
Y Togami
35549c0ddf
feat: support .xlsx (#218) 2022-10-28 18:46:09 -07:00
Ralf Northman
d5f71734d4
Implementation of response format config (#212) (#215) 2022-10-23 08:54:52 +00:00
Y Togami
ac43a36583
chore: typo (#214) 2022-10-22 12:43:59 -07:00
Y Togami
e9d0a03b89
dep: replace tempdir to tempfile (#202) 2022-10-15 17:53:42 -07:00
Y Togami
8180fb59d6
feat: Support jsonl for the file extension (#200)
* chore: add sample .ndjson .jsonl file

* test: add test for loading .ndjson and .jsonl

* feat: support jsonl for the file extension
2022-10-14 23:31:39 -07:00
Qingping Hou
4e812b9995 split database feature flag into database-mysql and database-sqlite 2022-09-25 18:58:05 -07:00
Qingping Hou
37e5d5b03a bump datafusion to 12 and sqlparser to 0.23 2022-09-25 18:58:05 -07:00
QP Hou
0d08aeb092
fix: support region and endpoint override for s3 (#187) 2022-08-24 22:42:28 -07:00
Qingping Hou
5972467cbb fix linter errors from new rustc release 2022-08-13 11:43:16 -07:00
Qingping Hou
2b03943480 fix: support loading database source in yaml config 2022-08-08 01:03:08 -07:00
QP Hou
193ea79878
support postgres wire protocol in query layer (#169) 2022-05-29 19:20:58 -07:00
QP Hou
6dc3b77a62
bump datafusion and sqlparser dep (#168) 2022-05-28 19:41:42 -07:00
zemel leong
3ace6078fa
Add MySQL, Sqlite support. (#162)
* added MySQL and Sqlite datasource support
* updated arrow, datafusion and deltalake to latest version
* cleared simd ci test cache to workaround nightly compiler bug
2022-04-07 00:55:10 -07:00
QP Hou
49aeaf1d71
add kvstore support (#161) 2022-04-02 19:46:58 -07:00
zemel leong
86b8cbf722
Add dynamic register and update table function. (#154)
* Add dynamic register and update table function.

* Modify Mutex to RwLock

Co-authored-by: zemel leong <zemel.leong@gmail.com>
2022-03-15 00:41:04 -07:00
QP Hou
9cdfb4a555
avoid schema_map deepcopy (#150) 2022-02-25 23:44:06 -08:00
Tiphaine Ruy
03f07b3b27
Fix simdtest case insensitive (#149)
Co-authored-by: eodyn <eodyn@eodyn.com>
2022-02-25 12:06:02 -08:00
QP Hou
24598db989
fix rust 1.59 clippy errors (#146) 2022-02-24 23:25:49 -08:00
Tiphaine Ruy
646c467916
chore: bump version for axum, dirs, sqlparser, yup-auth2, (#145)
* chore: bump version for axum, dirs, sqlparser, yup-auth2,

* chore: Bump datafusion to 7 and Delta-rs to f5e4b5f94393b517d3d88c200d7169cce11b304b

* fix(test): query are now case insensitive in datafusion

* fix(test): Remove println! statements
2022-02-24 22:57:03 -08:00
QP Hou
cd271c65ac
support empty cells in google spreadsheet (#141) 2022-02-14 22:23:17 -08:00
QP Hou
528f89efc9
fix json table partition loading (#133) 2022-01-27 20:14:11 -08:00
Jiayu Liu
badaecdf17
use impl into response (#131) 2022-01-26 17:41:19 -08:00
QP Hou
c900021f46
fix warnings from new rust release (#130) 2022-01-21 21:01:35 -08:00
yfaming
8127c8719d
make logical plan construction consistent and deterministic across query interfaces (#118)
close #115

Co-authored-by: yanfaming8701 <yanfaming8701@ipalfish.com>
2021-11-30 09:21:14 +00:00
QP Hou
c1ae60d667
make sort limit query deterministic for rest and graphql (#114)
* make sort limit query deterministic for rest and graphql
* bump version
2021-11-23 20:24:14 -08:00