Daniël Heres
f72299f142
Dependency upgrades ( #75 )
2021-09-13 19:12:45 +00:00
Thomas Peiselt
ea84099b07
Lazy load delta: Support for large tables ( #71 )
...
* Allow for delta tables to be directly backed by storage.
Enables experimental support for delta tables that are too large to be
stored in memory. We directly expose `DeltaTable` instead of copying the
data into a datafusion::Memtable.
Disadvantages:
- in the new mode, no support for S3
- as we're relying on datafusion to handle the parquet files directly,
nested schemas and certain data types may not work properly.
2021-09-06 00:56:55 -07:00
Erwin Kroon
ff2d06b0e4
add support for all Arrow IPC formats in roapi-http ( #67 )
...
* add support for all arrow IPC formats in roapi-http
* refactor: schema inferrence and partitions in 1 loop
2021-09-05 14:27:09 -07:00
Qingping Hou
3ff234798e
better error message for fs io source
2021-09-04 17:32:39 -07:00
tempbottle
ac761afea0
explicit windows platform support
2021-09-04 08:50:58 -07:00
Thomas Peiselt
5ace8b8695
Lazy load parquet ( #63 )
...
* PoC for parquet: reading a table by registering parquet directly.
* Adding config flag and restoring existing _in-memory_ code path.
* Addressing review comment: separate `to_mem_table()`.
* Addressing review comment: default-able `LoadOptionParquet`.
* Adding test: make sure we instantiated `datafusion::datasource::ParquetTable`
2021-09-03 19:36:18 +00:00
Erwin Kroon
c79a35a3fc
add parquet encoding ( #64 )
...
* add parquet encoding
* add alternative content-type for parquet encoding
2021-08-30 06:45:13 +00:00
Qingping Hou
0e8690d509
use reexported arrow from datafusion in columnq
2021-08-14 16:33:27 -07:00
Qingping Hou
d4a2ad9d84
bump datafusion to 5.0.0 release
2021-08-14 16:33:27 -07:00
Qingping Hou
0a2a965072
update arrow to 5.x and datafusion to latest
2021-08-08 17:42:12 -07:00
Qingping Hou
98110356d1
fix test
2021-07-22 23:38:28 -07:00
Qingping Hou
95f9ced23d
support deriving table name from table uri
2021-07-22 23:38:28 -07:00
Qingping Hou
ddf611cc0d
support csv serialization
2021-07-19 22:32:59 -07:00
Qingping Hou
7c51ff0fe4
moving encoding layer into columnq & support json cli output
2021-07-19 22:32:59 -07:00
Qingping Hou
d7fab6369c
bump deltalake version to 0.4
2021-07-18 17:59:32 -07:00
Qingping Hou
d5359b4e56
lots of goodies
...
* add in memory io source support
* add sql command to columnq cli
* support passing table format as columnq cli option
2021-07-18 00:06:59 -07:00
Qingping Hou
afb130b0f8
move columnq cli into its own package
2021-07-05 23:10:47 -07:00
Qingping Hou
9238850e3d
enable readline with history
2021-07-05 23:10:47 -07:00
Qingping Hou
ef82e24a64
support SHOW COLUMNS statement in columnq CLI
2021-07-05 23:10:47 -07:00
Qingping Hou
b376c56b7a
bump datafusion to latest version for full join support
2021-07-04 00:44:54 -07:00
Qingping Hou
4c78d1e728
update cargo install command to use lock file
2021-06-13 17:40:42 -07:00
Qingping Hou
3b415b2a5b
bump version to 0.1.3
2021-04-11 23:29:47 -07:00
Qingping Hou
56b1b6840b
update to latest arrow & add date and time column type support
2021-04-11 23:01:18 -07:00
Qingping Hou
9afd69b95b
fix clippy error
2021-03-28 12:22:11 -07:00
Qingping Hou
b730714b05
add columnq cli demo
2021-03-28 12:22:11 -07:00
Qingping Hou
b58ff2fc62
improve s3 credential handling
2021-03-20 22:31:30 -07:00
Qingping Hou
edefae4310
feat: add delta table support
2021-03-11 23:10:06 -08:00
Qingping Hou
8fd2f96ab8
add ndjson support
2021-03-07 13:41:18 -08:00
Qingping Hou
27772bcf4d
remove unwrap in s3 io
2021-03-07 13:41:18 -08:00
Qingping Hou
f33d496c76
add s3 support
2021-03-07 12:11:37 -08:00
Qingping Hou
538c54b57a
add sql group by query test
2021-02-28 21:38:43 -08:00
QP Hou
3631c2db14
update arrow and datafusion to latest master ( #13 )
2021-02-28 19:50:27 +00:00
QP Hou
717de08e67
fix clippy warnings ( #7 )
2021-02-21 09:24:49 +00:00
QP Hou
2c39fb13e7
support loading multiple partitions ( #6 )
2021-02-21 09:09:39 +00:00
QP Hou
2f4646bbb4
feat: support http store for parquet data source ( #5 )
2021-02-20 22:42:22 +00:00
QP Hou
f6dfc1faaf
feat: support https transport for csv data source ( #4 )
...
close #1
2021-02-20 09:47:11 +00:00
Qingping Hou
e0f213c316
split project into roapi-http and columnq crates
2021-02-17 13:46:20 -08:00
