英文:
Using flink sql client to submit sql query. How to I restore from checkpoint or savepoint
问题
I have started a local flink cluster using
./start-cluster.sh
I have started a local sql-client using
./sql-client.sh
I am able to submit sql statement in Flink SQL
terminal.
I have run Set 'state.checkpoints.dir' = 'file:///tmp/flink-savepoints-directory-from-set';
--> I can see checkpoint folder and getting created and updated when the sql job is running. ( sql job is reading from a kafka topic, does some joins and writing to another topic).
When I cancel the job from the flink UI and submit the sql again, the job does not restore from the state. ( I am basing this on the fact that the output or final sink, emits the same message on every restart, its like the job is reading the beginning of source topic again).
I have not shutdown the flink cluster or kafka cluster.
I have 2 questions
-
How I get the sql query to restore from state ?
-
Is there a way to use
flink run -s ...
command to submit sql query directly instead of packaging this as a jar ?
英文:
I have started a local flink cluster using
./start-cluster.sh
I have started a local sql-client using
./sql-client.sh
I am able to submit sql statement in Flink SQL
terminal.
I have run Set 'state.checkpoints.dir' = 'file:///tmp/flink-savepoints-directory-from-set';
--> I can see checkpoint folder and getting created and updated when the sql job is running. ( sql job is reading from a kafka topic, does some joins and writing to another topic).
When I cancel the job from the flink UI and submit the sql again, the job does not restore from the state. ( I am basing this on the fact that the output or final sink, emits the same message on every restart, its like the job is reading the beginning of source topic again).
I have not shutdown the flink cluster or kafka cluster.
I have 2 questions
-
How I get the sql query to restore from state ?
-
Is there a way to use
flink run -s ...
command to submit sql query directly instead of packaging this as a jar ?
答案1
得分: 0
这在https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sqlclient/#start-a-sql-job-from-a-savepoint文档中有记录:
SET 'execution.savepoint.path' = '/tmp/flink-savepoints/savepoint-cca7bc-bb1e257f0dab';
英文:
This is documented at https://nightlies.apache.org/flink/flink-docs-stable/docs/dev/table/sqlclient/#start-a-sql-job-from-a-savepoint
SET 'execution.savepoint.path' = '/tmp/flink-savepoints/savepoint-cca7bc-bb1e257f0dab';
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论