Dejadoc & code duplication
Les mainteneurs répondent en général sous 1 jour
Personne n'a encore pris cette issue.
Évaluation
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Accessibilité débutants
- 45/100
Piste de recherche
Start with the listed duplicate blocks in src/ast, src/dialect, and src/parser, then review the paired dialect test cases under tests/. The issue asks for cleanup based on dejadoc’s matches and explicitly says to preserve dialect-specific assertions. Done means consolidating appropriate repeated code or tests while retaining those cases; the broad list spans many parser areas.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
Over the past couple of weeks I have been working on dejadoc, which is a tool to find exact code duplications. It started out as a tool to find duplicated doctests, which is where it guarantees 100% accuracy, but then I just continued to have a few ideas and now it is a general code duplication tool specialized for Rust.
TL;DR: it does reduction to canonical form followed by execution graph isomorphisms for each context block larger than a given size.
I hope it is interesting, and will likely lead to some cleaup.
Ciao,
Luca
Parser, dialects, and AST
- Operator formatting arms in
OperatorClassItem::fmtandOperatorFamilyItem::fmt(54 tokens,5900b4f1).src/ast/ddl.rs:5061·src/ast/ddl.rs:5297 - Repeated blocks in
Expr::fmt(33 tokens,ea06ea64).src/ast/mod.rs:2151·src/ast/mod.rs:2165 -
PrimaryKeyConstraint::spanandUniqueConstraint::span(95 tokens,17ba4875).src/ast/table_constraints.rs:498·src/ast/table_constraints.rs:572 - Alias keyword handling in
MsSqlDialect::is_select_item_aliasandis_table_factor_alias(37 tokens,786b90d1).src/dialect/mssql.rs:148·src/dialect/mssql.rs:182 -
parse_infixinMySqlDialectandSparkSqlDialect(48 tokens,1d1e6c9e).src/dialect/mysql.rs:103·src/dialect/spark.rs:141 - Blocks in Snowflake
parse_create_tableandparse_create_database(31 tokens,131fed19).src/dialect/snowflake.rs:1021·src/dialect/snowflake.rs:1218 - Arms in
parse_expr_prefix_by_unreserved_word(32 tokens,ad7f100d).src/parser/mod.rs:1695·src/parser/mod.rs:1706 - Blocks in
parse_cache_table(103 tokens,54bd5405).src/parser/mod.rs:5658·src/parser/mod.rs:5691 - Blocks in
parse_function_argandparse_aggregate_function_arg(38 tokens,78e10075).src/parser/mod.rs:6391·src/parser/mod.rs:6455 - Blocks in
parse_create_operator_classandparse_operator_family_add_operator(39 tokens,427630ff).src/parser/mod.rs:7622·src/parser/mod.rs:11896 - Blocks in
parse_create_tableandparse_optimize_table(30 tokens,644cb5fa).src/parser/mod.rs:8978·src/parser/mod.rs:20621 - Blocks within
parse_optional_column_option_inner(37 tokens,9fb20c90).src/parser/mod.rs:9752·src/parser/mod.rs:9788 - Blocks in
parse_optional_column_option_innerandparse_optional_table_constraint(60 tokens,498e55f2).src/parser/mod.rs:9812·src/parser/mod.rs:10289
Tests
Any consolidation should retain the dialect-specific cases and assertions. A lexical match alone does not justify deleting coverage.
-
parse_join_constraint_unnest_aliasin BigQuery and PostgreSQL tests (68 tokens,bc48d2a7).tests/sqlparser_bigquery.rs:1786·tests/sqlparser_postgres.rs:6070 -
test_bigquery_trimandtest_duckdb_trim(85 tokens,b4f1ce3a).tests/sqlparser_bigquery.rs:2456·tests/sqlparser_duckdb.rs:860 -
parse_delimited_identifiersin ClickHouse and SQL Server tests (205 tokens,d533ff35).tests/sqlparser_clickhouse.rs:158·tests/sqlparser_mssql.rs:860 - Arms in
parse_delimited_identifiersacross five dialect test files (57 tokens,dd0c118a).tests/sqlparser_clickhouse.rs:165·tests/sqlparser_mssql.rs:867·tests/sqlparser_postgres.rs:4825·tests/sqlparser_redshift.rs:110·tests/sqlparser_snowflake.rs:1415 - Blocks in
parse_useacross six dialect test files (45 tokens,4a50dddb).tests/sqlparser_clickhouse.rs:1485·tests/sqlparser_databricks.rs:229·tests/sqlparser_duckdb.rs:817·tests/sqlparser_mssql.rs:1767·tests/sqlparser_mysql.rs:604·tests/sqlparser_snowflake.rs:3292 - First matching arms in
parse_json_objectandparse_mssql_json_array(30 tokens,6628f171).tests/sqlparser_common.rs:1841·tests/sqlparser_mssql.rs:1155 - Second matching arms in
parse_json_objectandparse_mssql_json_array(30 tokens,7f486c49).tests/sqlparser_common.rs:1857·tests/sqlparser_mssql.rs:1171 -
assert_mysql_query_valuearms in string-unescaping tests (37 tokens,dd3b9b9e).tests/sqlparser_common.rs:11801·tests/sqlparser_common.rs:11864 - First matching arms in
parse_create_table_likeandparse_create_table_like_with_defaults(37 tokens,9b33762e).tests/sqlparser_common.rs:19050·tests/sqlparser_postgres.rs:685 - Second matching arms in
parse_create_table_likeandparse_create_table_like_with_defaults(37 tokens,7eb8d06f).tests/sqlparser_common.rs:19067·tests/sqlparser_postgres.rs:703 -
test_select_wildcard_with_excludein DuckDB and Snowflake tests (106 tokens,bc66e513).tests/sqlparser_duckdb.rs:156·tests/sqlparser_snowflake.rs:1506 -
parse_drop_triggerin SQL Server and MySQL tests (36 tokens,885b1dde).tests/sqlparser_mssql.rs:2579·tests/sqlparser_mysql.rs:4362 -
parse_create_table_gencolin MySQL and SQLite tests (45 tokens,2f1129d7).tests/sqlparser_mysql.rs:1403·tests/sqlparser_sqlite.rs:318 - Blocks in
parse_grantandparse_revoke(30 tokens,0a0f9e04).tests/sqlparser_mysql.rs:4106·tests/sqlparser_mysql.rs:4149 - Blocks in
parse_create_indexandparse_create_index_with_predicate(67 tokens,ab693f4d).tests/sqlparser_postgres.rs:3008·tests/sqlparser_postgres.rs:3440 - Arms in
test_snowflake_create_table_with_aggregation_policy(33 tokens,57e152aa).tests/sqlparser_snowflake.rs:212·tests/sqlparser_snowflake.rs:232 - Arms in
test_snowflake_create_table_with_row_access_policy(33 tokens,f3df65a9).tests/sqlparser_snowflake.rs:252·tests/sqlparser_snowflake.rs:274 - Arms in
test_snowflake_create_table_with_tag(42 tokens,af56ee7c).tests/sqlparser_snowflake.rs:320·tests/sqlparser_snowflake.rs:341 - Blocks in
test_create_stage_with_file_formatandtest_copy_into_file_format(72 tokens,0a513c29).tests/sqlparser_snowflake.rs:2123·tests/sqlparser_snowflake.rs:2636·tests/sqlparser_snowflake.rs:2678 - Blocks in
test_create_stage_with_copy_optionsandtest_copy_into_copy_options(46 tokens,37e25c15).tests/sqlparser_snowflake.rs:2159·tests/sqlparser_snowflake.rs:2713 - Arms in
test_copy_into_file_format(78 tokens,88464afb).tests/sqlparser_snowflake.rs:2636·tests/sqlparser_snowflake.rs:2678 - Blocks in
test_snowflake_stage_object_names_into_locationandtest_snowflake_stage_object_names_into_table(37 tokens,cfc10449).tests/sqlparser_snowflake.rs:2754·tests/sqlparser_snowflake.rs:2777
Benchmarks
Review whether sharing setup would change what each benchmark measures.
- Closures in
parse_table_factor_paren_chainandparse_function_arg_call_chain(33 tokens,3c68771c).sqlparser_bench/benches/sqlparser_bench.rs:263·sqlparser_bench/benches/sqlparser_bench.rs:287
- Langage dominant
- Rust
- Étoiles
- 3.5k
- Forks
- 805
- Merge moyen
- 2 j 3 h
- PR mergées (30 j)
- 116
Préparer son environnement
- Aucun Dockerfile ni fichier Docker Compose
- Propose un modèle de pull request
- Aucun guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de apache/datafusion-sqlparser-rs
-
Redshift
Difficulté 2/5 1-3 heures Accessibilité débutants 82/100
apache/datafusion-sqlparser-rs#2959 ·
Les mainteneurs répondent en général sous 1 jour
-
Redshift
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
apache/datafusion-sqlparser-rs#2958 ·
Les mainteneurs répondent en général sous 1 jour
-
PostgreSQL
Difficulté 2/5 1-3 heures Accessibilité débutants 76/100
apache/datafusion-sqlparser-rs#2862 ·
Les mainteneurs répondent en général sous 1 jour
-
PostgreSQL
Difficulté 2/5 1-3 heures Accessibilité débutants 75/100
apache/datafusion-sqlparser-rs#2861 ·
Les mainteneurs répondent en général sous 1 jour
-
PostgreSQL
Difficulté 2/5 1-3 heures Accessibilité débutants 77/100
apache/datafusion-sqlparser-rs#2858 ·
Les mainteneurs répondent en général sous 1 jour
Toutes les issues de apache/datafusion-sqlparser-rs
Issues similaires
-
area:schema documentation
Difficulté 2/5 1-3 heures Accessibilité débutants 76/100
Les mainteneurs répondent en général sous 1 jour
-
Difficulté 2/5 1-3 heures Accessibilité débutants 82/100
ElectronicWave/Elemental#252 ·
Les mainteneurs répondent en général sous 1 jour
-
bug
Difficulté 2/5 1-3 heures Accessibilité débutants 78/100
dekobon/big-code-analysis#1887 ·
Les mainteneurs répondent en général sous 1 jour
-
Shell中文符号输入异常Ouvertebug
Difficulté 2/5 1-3 heures Accessibilité débutants 76/100
Les mainteneurs répondent en général sous 1 jour
-
Difficulté 1/5 Moins d'une heure Accessibilité débutants 82/100
Les mainteneurs répondent en général sous 1 jour