[TASK][MEDIUM] JDBC result format negotiation for Arrow or Thrift

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before creating

• I have searched in the task list and found no similar tasks.

Mentor

• I have sufficient knowledge and experience of this task, and I volunteer to be the mentor of this task to guide contributors to complete the task.

Skill requirements

• Knowledge about JDBC
• Basic Knowledge with Scala Programming

Background and Goals

Arrow is one of supported JDBC result format which helps to maximize the fetching performance with verticalization, while Thrift is remained as default result format. Arrow requires Kyuubi Hive driver support. Currently, the result format can be switched via kyuubi.operation.result.format config, by session configs from client side, or can be set on server side configs. This requires pre-connection assertions and lack of flexibility for both client and server side. To introduce result format negotiation, the server provides an mechanism to minimize the gap for prefered result set.

Implementation steps

• Add preferred result format list on server side config. For example. arrow, thrift implies that server will prefer to use Arrow if no client preferred format information, while thrift is also support.
• And if client set kyuubi.operation.result.format config, it means the server tries to follow the dedicated result format. When the server fails to lookup targeted result format in supported list config, the server should throws unsupported exceptions.

Additional context

Original reporter is @bowenliang123