Extend Databricks Operators to DBFS Interaction

Description

Create operators and Hook to interact with Databricks' DBFS (https://docs.databricks.com/api/workspace/dbfs)

Use case/motivation

As per latest databricks plugin (https://github.com/apache/airflow/tree/main/airflow/providers/databricks) there is no possibility to interact with DBFS API.

As I had to do it in my job (and I have it quite developed), I thought it'd be a good idea to share it with the community

So far, I've got:

• An operator that uploads files to DBFS
• A hook that interacts with the DBFS API, respecting Databricks' Hooks logic and inheriting from BaseDatabricksHook

As part of the PR, I'd add:

• Some more operators (getting files, getting files metadata, deleting files)
• Tests in line with Airflow's test suite

Please LMK if you consider this a relevant contribution or not

Related issues

As one of the DBFS API endpoints uses PUT as verb., I'd need to include a modification in BaseDatabricksHook, because it is not supporting PUT ATM (see https://github.com/apache/airflow/blob/main/airflow/providers/databricks/hooks/databricks_base.py#L584)

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct