Commit Graph

161 Commits

Author SHA1 Message Date
QP Hou
c5efa9716d
upgrade dependencies (#425) 2026-03-24 00:51:44 -07:00
Qingping Hou
850df66fe2 refactor(columnq): box large error variants to fix clippy::result_large_err 2026-03-22 13:58:50 -07:00
prabodh516
63d51cfcdd
fix azure scheme (#410)
Some checks failed
build / build (push) Has been cancelled
build / build_ui (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
Current supported azure schemes are

` "az" | "adl" | "adfs" | "adfss" | "azure" =>
Ok(BlobStoreType::Azure),`

Updated azure scheme as

` "az" | "adl" | "abfs" | "abfss" | "azure" =>
Ok(BlobStoreType::Azure),`

Co-authored-by: Prabodh Mitra <prabodh.mitra@MediaKind.com>
2025-10-17 17:43:45 +00:00
Qingping Hou
162dbfe5ad fix clippy errors
Some checks failed
build / build (push) Has been cancelled
build / build_ui (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
Scheduled security audit / audit (push) Has been cancelled
2025-06-28 08:39:02 -07:00
Qingping Hou
f01f6b4d9e bump dependencies
Some checks failed
build / build (push) Has been cancelled
build / build_ui (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
2025-06-21 23:17:45 -07:00
Qingping Hou
4340b2224c fix cargo fmt issues 2025-06-15 20:19:08 -07:00
Qingping Hou
d33c80a9e5 fix clippy error 2025-06-15 20:17:19 -07:00
QP Hou
083a31d6cc
bump deltalake, datafusion and arrow to latest (#396) 2025-05-04 23:32:59 -07:00
QP Hou
1607c9a452
bump convergence to support string type (#394)
Some checks failed
build / build (push) Has been cancelled
build / build_ui (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
2025-04-23 22:57:44 -07:00
Mike Shauneu
6fa208fce4
Add HTTP object_store to be able to connect without caching in memory… (#393)
Before: 
```
roapi -t taxi=https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet,use_memory_table=false,format=parquet
[2025-04-23T22:43:25Z INFO  roapi::context] loading `uri(https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet)` as table `taxi`
error: Failed to build context: Failed to load table: Table error: Failed to infer table schema: Internal error: No suitable object store found for https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet. See `RuntimeEnv::register_object_store`.
This was likely caused by a bug in DataFusion's code and we would welcome that you file an bug report in our issue tracker
```

After:
```
roapi -t 'taxi=https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet,use_memory_table=false,format=parquet'`
[2025-04-23T22:47:00Z INFO  roapi::context] loading `uri(https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet)` as table `taxi`
[2025-04-23T22:47:00Z INFO  roapi::context] registered `uri(https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2024-01.parquet)` as table `taxi`
[2025-04-23T22:47:00Z INFO  roapi::startup] Running in read-only mode.
[2025-04-23T22:47:00Z INFO  roapi::startup] 🚀 Listening on 127.0.0.1:5432 for Postgres traffic...
[2025-04-23T22:47:00Z INFO  roapi::startup] 🚀 Listening on 127.0.0.1:32010 for FlightSQL traffic...
[2025-04-23T22:47:00Z INFO  roapi::startup] 🚀 Listening on 127.0.0.1:8080 for HTTP traffic...
```
2025-04-23 22:19:50 -07:00
QP Hou
b2d584a74e
fix postgres extension parsing (#388)
fixes https://github.com/roapi/roapi/issues/387
2025-03-23 18:22:32 -07:00
QP Hou
9ebc252011
remove lazy_static dep (#376)
Some checks failed
build / build (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
2025-01-21 22:46:17 -08:00
QP Hou
ba6288afe4
optimize: async table reload (#373)
closes https://github.com/roapi/roapi/issues/339
2025-01-04 21:45:33 -08:00
QP Hou
3d6ffa796c
feat: support specifying table name in database table load cfg (#371)
Some checks failed
build / build (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
2025-01-04 01:57:34 -08:00
QP Hou
809521124f
change default refresh interval to 1 hour (#370) 2025-01-04 01:34:22 -08:00
QP Hou
1570c7d4bb
update minor dependency versions & fix release (#366)
Some checks failed
build / build (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
2025-01-01 01:54:02 -08:00
QP Hou
a9317f73eb
cleanup & update dependencies tree (#365)
update datafusion to 43, deltalake to 0.22, axum to 0.7 and arrow to 53

---------

Co-authored-by: hozan23 <hozan23@karyontech.net>
2024-12-31 19:06:49 -08:00
QP Hou
6e6b315ca3
add header override support for http backed sources (#364)
closes #358
2024-12-31 01:22:23 -08:00
QP Hou
49da15e4c0
refactor: make table refresher a required field (#360)
other drive by changes:
* add nix flake for quick dev env setup
2024-12-28 22:44:34 -08:00
QP Hou
c1e26a84d1
enable continuous background refresh for delta tables (#352)
Some checks failed
build / build (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
2024-12-02 06:49:16 -08:00
Rui Chen
85c55c9951
fix: add missing explicit lifetime in ExcelSubrange return type for rust 1.83 build (#353)
seeing some regression build failure when building roapi 0.12.0 against
rust 1.83.0

```
  error: elided lifetime has a name
    --> columnq/src/table/excel.rs:48:10
     |
  41 | impl<'a> ExcelSubrange<'a> {
     |      -- lifetime `'a` declared here
  ...
  48 |     ) -> ExcelSubrange {
     |          ^^^^^^^^^^^^^ this elided lifetime gets resolved as `'a`
     |
  note: the lint level is defined here
    --> columnq/src/lib.rs:1:9
     |
  1  | #![deny(warnings)]
     |         ^^^^^^^^
     = note: `#[deny(elided_named_lifetimes)]` implied by `#[deny(warnings)]`
```

relates to https://github.com/Homebrew/homebrew-core/pull/199379

Signed-off-by: Rui Chen <rui@chenrui.dev>
2024-12-02 08:51:29 +00:00
hozan23
daabd84418
columnq: add apply_query() and exec_query_with_df() to query::rest module (#349)
Following this PR #348 

This PR exposes two functions from the columnq::query::rest module:

    apply_query()
    exec_query_with_df()

This PR also includes breaking changes to ensure consistency with the
function names in the `columnq::query::graphql` module. I renamed the
function `query_table` to `exec_table_query`.
2024-10-20 17:14:01 -07:00
hozan23
e282c6c1d1
columnq: add apply_query() and exec_query_with_df() to query::graphql module (#348)
This PR exposes two functions from the `columnq::query::graphql` module:

- `apply_query()`
- `exec_query_with_df()`

These functions are similar to `query_to_df()` and `exec_query()`, but
they accept a `DataFrame` instead of a `SessionContext`.

This is useful for defining rules directly within a `DataFrame` without
modifying the table schema in DataFusion's session context.
2024-10-20 17:09:06 -07:00
Brendan Ryback
b776dd34ce
Support for Dynamic Table Deletion #340 (#343)
Some checks failed
build / build (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
closes #340
2024-08-25 21:02:56 +00:00
QP Hou
08ec089a84
bump object_store and, arrow (52) and datafusion (39) (#341)
Some checks failed
build / build (push) Has been cancelled
build / database_test (push) Has been cancelled
build / object_store_memory_test (push) Has been cancelled
build / object_store_direct_test (push) Has been cancelled
build / openssl_build (push) Has been cancelled
build / mac_cross_build (push) Has been cancelled
build / Docker Image Build (push) Has been cancelled
columnq-cli release / Validate git tag (push) Has been cancelled
roapi release / Validate git tag (push) Has been cancelled
columnq-cli release / macos (push) Has been cancelled
columnq-cli release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
columnq-cli release / PyPI Release (push) Has been cancelled
roapi release / macos (push) Has been cancelled
roapi release / windows (map[features:database-sqlite python-architecture:x64 target:x86_64-pc-windows-msvc]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:aarch64-musl manylinux:2014 name_suffix: rustflags: target:aarch64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / linux (map[features:rustls,database-sqlite image_tag:x86_64-musl manylinux:2010 name_suffix: rustflags:-C target-cpu=skylake target:x86_64-unknown-linux-musl upload:true]) (push) Has been cancelled
roapi release / PyPI Release (push) Has been cancelled
roapi release / Docker Image Release (push) Has been cancelled
close https://github.com/roapi/roapi/issues/337
2024-08-04 20:18:35 -07:00
QP Hou
3bfcb29bcd
bump datafusion to 37 and arrow to 51 (#334)
closes #324 #325
2024-05-26 23:32:56 -07:00
QP Hou
9eba0281c6
support http for delta tables (#333) 2024-05-26 00:33:01 -07:00
QP Hou
1938fe6c2b
fix remote storage support for deltalake (#331) 2024-05-19 20:56:46 -07:00
QP Hou
5a0b68de6d
fix: exclude folder in object_store s3 list content (#329) 2024-05-05 17:26:06 -07:00
QP Hou
ffc794c43b
support partition filter pushdown (#326) 2024-04-28 00:03:34 -07:00
QP Hou
199f9d8e0f
bump datafusion to 35 and arrow to 50 (#320)
also bump rustc version
2024-03-24 15:58:36 -07:00
QP Hou
c2a3268d44
add token & basic auth support for flightsql endpoint (#318)
required to enable SSL in grafana's flightsql plugin
2024-03-10 22:48:01 -07:00
Maksym Dovhal
81a25205b7
Update xlsx table (#316)
This PR is a fix for issue https://github.com/roapi/roapi/issues/259

List of updates/fixes:
* module xlsx renamed to excel.
* Allow reading not only xlsx format but also xls, ods, xlsb
* Allow Excel DateTime format and transform it to arrow
Timestamp(Seconds, None)
* Allow using NULLs in any data types and use null value instead of
string "null"
* Fix issue with incorrect data type inference when multiple data types
are detected.
* Add possibility to specify data schema in config.
* Add new options: -
rows_range_start
 - rows_range_end
 - columns_range_start
 - columns_range_end
 - schema_inference_lines
* Make sheet_name optional and if it is not specified than use first
sheet by default

* Bump calamine crate to version 0.23.1 and add feature "dates"
(supporting for DateTime column format)

Documentation updates: https://github.com/roapi/docs/pull/20
2024-01-31 20:16:34 -08:00
QP Hou
9377b7ccad
support infer schema by list of files (#310) 2023-11-19 20:14:26 -08:00
QP Hou
a322d1b443
support defining nested enum data types in schema config (#308) 2023-11-18 19:01:53 -08:00
QP Hou
b70c3bcb0b
enable aws feature in object_store crate by default 2023-10-31 01:03:22 -07:00
QP Hou
d531f8ae46
support partitioned tables (#307) 2023-10-20 16:00:07 -07:00
QP Hou
ea01ffe0d3
upgrade clap to 4 (#304) 2023-10-14 21:09:21 -07:00
QP Hou
ef113c5521
add flightsql support (#301) 2023-10-10 22:31:14 -07:00
QP Hou
b10afb0116
enable resolve 2 in cargo.toml (#300) 2023-10-03 01:53:06 -07:00
QP Hou
eef379f9bb
switch to snafu for error handling in columnq (#299) 2023-10-03 00:27:04 -07:00
QP Hou
2baf669841
update datafusion to 31 (#297) 2023-09-30 19:24:47 -07:00
Joe
ffd1ca6dda
remove unwrap (#294) 2023-09-11 22:42:50 -07:00
Joe
5348f68d6e
refactor read table from db code (#293) 2023-09-10 23:42:28 -07:00
Joe
ff3617767d
feat: support postgresql (#286) 2023-08-29 09:06:11 -07:00
Charles Swanberg
01c5422206
Cloud support deltalake (#280)
* fix: enable cloud support for deltalake

* fix clippy issues

* formatting fixes
2023-06-20 00:43:35 -07:00
QP Hou
df39a13d3f
upgrade datafusion and arrow (#279)
* update arrow, datafusion, delta, convergence, object_store, sqlparser

* fix breaking code

* update to datafusion 23

* update connectorx fork

---------

Co-authored-by: Chitral Verma <chitralverma@gmail.com>
2023-05-29 23:25:45 -07:00
Jiayu Liu
ae13d3bc9e
support toml as config lang (#277)
* support toml as config lang

* update readme
2023-05-22 22:45:39 -07:00
Robert Gieseke
411d727263 Update connector-x to latest commit of original repo
Closes #271
2023-04-14 08:27:30 -07:00
Akshith Madhur
4cd9360fb4
fixed csv-s3 issue (#268)
Co-authored-by: Elliot Alderson <elliot@Elliots-MacBook-Pro.local>
2023-03-14 23:35:02 -07:00