CodeCow/roapi
1
0
Fork 0
mirror of https://github.com/roapi/roapi.git synced 2026-06-05 21:04:02 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
127 Commits 3 Branches 47 Tags 2.2 MiB
roapi-http-v0.5.0
Commit Graph

59 Commits

Author SHA1 Message Date
QP Hou
ede97cb05f
change default batch size to 8192 (#112) 2021-11-21 19:12:25 -08:00
QP Hou
928027fe31
bump datafusion to 6.x (#111) 
adds support for accessing nested struct field and array element
 2021-11-21 14:41:02 -08:00
Toby Hede
494f5d5b13
Wrap reader as bufreader for file json and ndjson (#109) 
* wrap read as bufreader for json
* wrap read as bufreader for ndjson
 2021-11-21 06:29:03 +00:00
Toby Hede
9e3769bb8c
make batch_size configurable (#107) 2021-11-19 00:54:37 +00:00
Charlie Harrington
65702fd350
fix typo 'GoogleSpeadsheet' -> 'GoogleSpreadsheet' (#108) 2021-11-16 19:11:18 +00:00
Erwin Kroon
56b8d0b8d3
Fallback to application/json encoding (#104) 
* feat: fallback to application/json encoding

* fix: update tower-http git hash

* Update roapi-http/src/api/mod.rs

* build: clean roapi-http output in default build

* fix: tower-http cors Any -> any()

Co-authored-by: QP Hou <dave2008713@gmail.com>
 2021-11-13 11:06:58 -08:00
QP Hou
de1359f150
support custom s3 endpoint (#99) 
* mention uri encoding in url parse error

* support custom s3 endpoint
 2021-10-30 23:28:11 -07:00
QP Hou
6ce319de5f
add native-tls feature to support linking with system provided openssl (#93) 
other ci related changes:
* temporarily disable openssl build in release jobs
* upload windows build to github release
 2021-10-14 09:05:54 -07:00
QP Hou
7776462d3c
optimize release build (#88) 
* switching to snmalloc yields 20% performance boost
* codegen-unit=1 yields 20% performance boost
 2021-10-10 17:08:07 -07:00
QP Hou
fd824cabf2
validate docker build in CI (#87) 2021-10-10 13:46:17 -07:00
Erwin Kroon
92e1e2408e
Switch to rustls as the default (#83) 
Added the new features to the linux build-matrix and the windows and macos build only build the default rustls feature. Only the rustls matrix-items will have their wheels and artifacts publiced.
 2021-09-30 09:45:33 -07:00
Erwin Kroon
2183332de9
Enable SIMD feature for docker build (#81) 
* Enable SIMD feature for docker build

Fixes #15

* Force cargo-cache installation
 2021-09-26 10:14:53 -07:00
Qingping Hou
509af14f61 fix tests 2021-09-19 18:59:11 -07:00
Qingping Hou
aa36d64dce bump datafusion, arrow and deltalake version 2021-09-19 18:37:56 -07:00
Daniël Heres
f72299f142
Dependency upgrades (#75) 2021-09-13 19:12:45 +00:00
Qingping Hou
5696e3f855 add columnq-cli release job 2021-09-06 00:58:03 -07:00
Thomas Peiselt
ea84099b07
Lazy load delta: Support for large tables (#71) 
* Allow for delta tables to be directly backed by storage.

Enables experimental support for delta tables that are too large to be
stored in memory. We directly expose `DeltaTable` instead of copying the
data into a datafusion::Memtable.

Disadvantages:
- in the new mode, no support for S3
- as we're relying on datafusion to handle the parquet files directly,
  nested schemas and certain data types may not work properly.
 2021-09-06 00:56:55 -07:00
Erwin Kroon
ff2d06b0e4
add support for all Arrow IPC formats in roapi-http (#67) 
* add support for all arrow IPC formats in roapi-http

* refactor: schema inferrence and partitions in 1 loop
 2021-09-05 14:27:09 -07:00
Qingping Hou
3ff234798e better error message for fs io source 2021-09-04 17:32:39 -07:00
tempbottle
ac761afea0 explicit windows platform support 2021-09-04 08:50:58 -07:00
Thomas Peiselt
5ace8b8695
Lazy load parquet (#63) 
* PoC for parquet: reading a table by registering parquet directly.

* Adding config flag and restoring existing _in-memory_ code path.

* Addressing review comment: separate `to_mem_table()`.

* Addressing review comment: default-able `LoadOptionParquet`.

* Adding test: make sure we instantiated `datafusion::datasource::ParquetTable`
 2021-09-03 19:36:18 +00:00
Erwin Kroon
c79a35a3fc
add parquet encoding (#64) 
* add parquet encoding

* add alternative content-type for parquet encoding
 2021-08-30 06:45:13 +00:00
Qingping Hou
0e8690d509 use reexported arrow from datafusion in columnq 2021-08-14 16:33:27 -07:00
Qingping Hou
d4a2ad9d84 bump datafusion to 5.0.0 release 2021-08-14 16:33:27 -07:00
Qingping Hou
075fc70754 bump version to 0.3.4 2021-08-08 21:32:15 -07:00
Qingping Hou
0a2a965072 update arrow to 5.x and datafusion to latest 2021-08-08 17:42:12 -07:00
Qingping Hou
98110356d1 fix test 2021-07-22 23:38:28 -07:00
Qingping Hou
95f9ced23d support deriving table name from table uri 2021-07-22 23:38:28 -07:00
Qingping Hou
e5989d1f9d bump arrow for struct support in table output 2021-07-21 00:04:18 -07:00
Qingping Hou
ddf611cc0d support csv serialization 2021-07-19 22:32:59 -07:00
Qingping Hou
7c51ff0fe4 moving encoding layer into columnq & support json cli output 2021-07-19 22:32:59 -07:00
Qingping Hou
d7fab6369c bump deltalake version to 0.4 2021-07-18 17:59:32 -07:00
Qingping Hou
232f3280e4 bump version in cargo to 0.3.1 2021-07-18 00:45:14 -07:00
Qingping Hou
d5359b4e56 lots of goodies 
* add in memory io source support
* add sql command to columnq cli
* support passing table format as columnq cli option
 2021-07-18 00:06:59 -07:00
Qingping Hou
afb130b0f8 move columnq cli into its own package 2021-07-05 23:10:47 -07:00
Qingping Hou
9238850e3d enable readline with history 2021-07-05 23:10:47 -07:00
Qingping Hou
ef82e24a64 support SHOW COLUMNS statement in columnq CLI 2021-07-05 23:10:47 -07:00
QP Hou
b332034d0a
bump to datafusion fork for join column fixes (#47) 2021-07-04 07:57:08 +00:00
Qingping Hou
44ba6364d2 install arrow from git source 2021-07-04 00:44:54 -07:00
Qingping Hou
34cfca9107 bump cargo versions 2021-07-04 00:44:54 -07:00
Qingping Hou
b376c56b7a bump datafusion to latest version for full join support 2021-07-04 00:44:54 -07:00
Qingping Hou
4c78d1e728 update cargo install command to use lock file 2021-06-13 17:40:42 -07:00
QP Hou
f0f68f999c
bump arrow & datafusion to 4.x (#44) 
* bump arrow & datafusion to 4.x

* pin arrow to 4.3.0 for simd nightly build fix
 2021-06-13 06:38:47 +00:00
Qingping Hou
3b415b2a5b bump version to 0.1.3 2021-04-11 23:29:47 -07:00
Qingping Hou
56b1b6840b update to latest arrow & add date and time column type support 2021-04-11 23:01:18 -07:00
Qingping Hou
9afd69b95b fix clippy error 2021-03-28 12:22:11 -07:00
Qingping Hou
b730714b05 add columnq cli demo 2021-03-28 12:22:11 -07:00
Qingping Hou
b58ff2fc62 improve s3 credential handling 2021-03-20 22:31:30 -07:00
Qingping Hou
edefae4310 feat: add delta table support 2021-03-11 23:10:06 -08:00
Qingping Hou
8fd2f96ab8 add ndjson support 2021-03-07 13:41:18 -08:00