lakefs 的presto on s3a

lakefs

https://lakefs.io/blog/the-quick-guide-for-running-presto-locally-on-s3/#skip-to-the-example

执行操作的环境为：https://github.com/huangxiaofeng10047/blog-presto-local

mini 环境参考的是：git@github.com:treeverse/lakeFS-samples.git

启动的是 lakefs 的客户端。

启动 presto 客户端

docker-compose exec presto presto-cli --catalog s3 --schema default

1
2


CREATE SCHEMA main
WITH (location = 's3a://example/main')

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


CREATE TABLE main.page_views (
  view_time timestamp,
  user_id bigint,
  page_url varchar,
  ds date,
  country varchar
)
WITH (
  format = 'ORC',
  partitioned_by = ARRAY['ds', 'country'],
  bucketed_by = ARRAY['user_id'],
  bucket_count = 50
)；
CREATE TABLE main.request_logs (
  request_time timestamp,
  url varchar,
  ip varchar,
  user_agent varchar
)
WITH (
  format = 'TEXTFILE',
  external_location = 's3a://example/main/data/logs/'
)
;

1

 insert into main.page_views select from_unixtime(1584660148),12,'http://localhost:8080',current_date,'cn';

注意插入数据只在 hive 这边能写入，presto 这边写入不了。