Drill fail to describe postgres schema

Open

#2,855 opened on Dec 5, 2023 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty: 4/5
Estimated time: 3-5 days
Newbie friendliness: 42/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Tech stack: java, postgresql
Domain: databases

Research direction

Reproduce DESCRIBE SCHEMA test_postgres.data using the PostgreSQL storage configuration, then start at DescribeSchemaHandler.getPlan in the stack trace and inspect the schema-description path. Done means the query completes and describes the schema without the Jackson passwordFilter error; add or update a regression test if the existing test structure identifies one.

Written by the indexing model from the issue text.

Description

bug

Lab Configuration:
drill version: 1.21.1

drill postgres storage plugin configuration:

{
  "type": "jdbc",
  "driver": "org.postgresql.Driver",
  "url": "jdbc:postgresql://<ip>:<port>/data",
  "credentialsProvider": {
    "credentialsProviderType": "VaultCredentialsProvider",
    "secretPath": "drill/test_postgres",
    "propertyNames": {
      "username": "usernameSecret",
      "password": "passwordSecret"
    }
  },
  "authMode": "SHARED_USER",
  "writerBatchSize": 10000,
  "enabled": true
}

Test on drill portal: http://:8047/query
1.
show schemas
output is something like below:

<html>
<body>
<!--StartFragment-->
SCHEMA_NAME
--
sys
<html>
<body>
<!--StartFragment-->
test_postgres.information_schema
test_postgres.public
test_postgres.tiger
test_postgres.tiger_data
test_postgres.topology
test_postgres.data
test_postgres
<!--EndFragment-->
</body>
</html>

<!--EndFragment-->
</body>
</html>

describe schema on any of postgres schema with below command
DESCRIBE SCHEMA|DATABASE [.workspace]

describe schema test_postgres.data

we get below error on portal:

error on drill docker container:

	at org.apache.drill.exec.server.rest.RestQueryRunner.submitQuery(RestQueryRunner.java:99)
	at org.apache.drill.exec.server.rest.RestQueryRunner.run(RestQueryRunner.java:54)
	at org.apache.drill.exec.server.rest.QueryResources.submitQuery(QueryResources.java:161)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
	at org.apache.drill.exec.server.rest.header.ResponseHeadersSettingFilter.doFilter(ResponseHeadersSettingFilter.java:71)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at org.apache.drill.exec.server.rest.CsrfTokenValidateFilter.doFilter(CsrfTokenValidateFilter.java:55)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at org.apache.drill.exec.server.rest.CsrfTokenInjectFilter.doFilter(CsrfTokenInjectFilter.java:54)
	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:571)
	at org.apache.drill.exec.server.rest.auth.DrillHttpSecurityHandlerProvider.handle(DrillHttpSecurityHandlerProvider.java:163)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:516)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
	at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception during fragment initialization: Cannot resolve PropertyFilter with id 'passwordFilter'; no FilterProvider configured
	at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:301)
	at .......(:0)
Caused by: java.lang.IllegalArgumentException: Cannot resolve PropertyFilter with id 'passwordFilter'; no FilterProvider configured
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4449)
	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:4380)
	at org.apache.drill.exec.planner.sql.handlers.DescribeSchemaHandler.getPlan(DescribeSchemaHandler.java:100)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:298)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:179)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:129)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:94)
	at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:594)
	at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274)
	... 1 common frames omitted
Caused by: java.lang.Exception: Cannot resolve PropertyFilter with id 'passwordFilter'; no FilterProvider configured
	at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
	at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1306)
	at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:408)
	at com.fasterxml.jackson.databind.ser.std.StdSerializer.findPropertyFilter(StdSerializer.java:426)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFieldsFiltered(BeanSerializerBase.java:814)
	at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeWithType(BeanSerializerBase.java:655)
	at com.fasterxml.jackson.databind.ser.impl.TypeWrappedSerializer.serialize(TypeWrappedSerializer.java:32)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4427)
	... 9 common frames omitted

Thanks for support.

Dominant language: Java
Stars: 2k
Forks: 989
Avg merge: 3d 14h
Merged PRs (30d): 7

Getting set up

Ships a Dockerfile or Docker Compose file
Has a pull request template
No contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

Java 8 JDKs get no proxy CA: app-user-init.sh only looks for lib/security/cacerts
Open
- Is something wrong?
Difficulty 1/5 Under an hour Newbie friendliness 85/100

VirtusLab/sandcat#135 · Opened Oct 6, 2026

Maintainers usually reply within 1 day
Unclosed link reference definition title keeps the partial title and the following paragraph's source spans (0.30.0)
Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 70/100

commonmark/commonmark-java#460 · Opened Oct 6, 2026
General-purpose Instant-to-Timestamp converter misidentifies Instant.EPOCH as COMMIT_TIMESTAMP sentinel
Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 72/100

GoogleCloudPlatform/spring-cloud-gcp#4664 · Opened Oct 6, 2026

Maintainers usually reply within 1 day
As a user I want a reference and explanation on how the program works on the website
Open
- Is something wrong?
enhancement user story

Difficulty 2/5 1-3 hours Newbie friendliness 70/100

Lucaa0312/LeoPlaner#157 · Opened Oct 6, 2026
SQLite: AssertionError on CREATE TABLE with a generated column loop (SQLite ≥ 3.44)
Open
- Is something wrong?
Difficulty 2/5 1-3 hours Newbie friendliness 86/100

sqlancer/sqlancer#1369 · Opened Oct 6, 2026

More Java issues

Drill fail to describe postgres schema

Is something wrong?

Assessment

Research direction

Description

Getting set up

First steps

More from apache/drill

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Similar issues

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Is something wrong?

Get new issues in your inbox