Accessing Flume logs on the Windows 11 platform

Start with org.apache.flume.source.taildir.ReliableTaildirEventReader.getInode at line 284 and reproduce the failure using the provided Windows 11 configuration and startup command. Check the Taildir source startup path and verify completion when the configured log files can be monitored without the “View 'unix' not available” exception.

My log configuration file:

a1.sources = r1
a1.channels = c1
a1.sinks = k1

Describe/configure the source

a1.sources.r1.type = TAILDIR
#断点续传
#a1.sources.r1.positionFile = D:\flumedev\tail_dir.json

多目录监控

a1.sources.r1.filegroups = f1 f2
a1.sources.r1.filegroups.f1 = D:\flumedev\source\test2\asdfile.log
a1.sources.r1.filegroups.f2 = D:\flumedev\source\test1\test.log

channal

a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

Describe the sink

a1.sinks.k1.type = logger

Bind the source and sink to the channel

a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

My startup Flume command:
D:\apache-flume-1.11.0-bin>bin\flume-ng agent --conf conf --conf-file D:\flumedev\flume-taildir-hdfs.conf --name a1

The error logs I received:
17 七月 2024 00:00:01,709 INFO [lifecycleSupervisor-1-4] (org.apache.flume.source.taildir.TaildirSource.start:94) - r1 TaildirSource source starting with directory: {f1=D:\flumedev\source\test2\asdfile.log, f2=D:\flumedev\source\test1\test.log}
17 七月 2024 00:00:01,709 INFO [lifecycleSupervisor-1-4] (org.apache.flume.source.taildir.ReliableTaildirEventReader.:84) - taildirCache: [{filegroup='f1', filePattern='D:\flumedev\source\test2\asdfile.log', cached=true}, {filegroup='f2', filePattern='D:\flumedev\source\test1\test.log', cached=true}]
17 七月 2024 00:00:01,712 INFO [lifecycleSupervisor-1-4] (org.apache.flume.source.taildir.ReliableTaildirEventReader.:85) - headerTable: {}
17 七月 2024 00:00:01,712 ERROR [lifecycleSupervisor-1-4] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:251) - Unable to start PollableSourceRunner: { source:Taildir source: { positionFile: C:/Users/liu970/.flume/taildir_position.json, skipToEnd: false, byteOffsetHeader: false, idleTimeout: 120000, writePosInterval: 3000 } counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.UnsupportedOperationException: View 'unix' not available
at sun.nio.fs.AbstractFileSystemProvider.readAttributes(AbstractFileSystemProvider.java:91) ~[?:1.8.0_43]
at java.nio.file.Files.readAttributes(Files.java:1964) ~[?:1.8.0_43]
at java.nio.file.Files.getAttribute(Files.java:1869) ~[?:1.8.0_43]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.getInode(ReliableTaildirEventReader.java:284) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.updateTailFiles(ReliableTaildirEventReader.java:248) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.(ReliableTaildirEventReader.java:93) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.(ReliableTaildirEventReader.java:49) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader$Builder.build(ReliableTaildirEventReader.java:355) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.TaildirSource.start(TaildirSource.java:105) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:71) ~[flume-ng-core-1.11.0.jar:1.11.0]
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249) ~[flume-ng-core-1.11.0.jar:1.11.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_43]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_43]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_43]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_43]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_43]
17 七月 2024 00:00:04,724 INFO [lifecycleSupervisor-1-3] (org.apache.flume.source.taildir.TaildirSource.start:94) - r1 TaildirSource source starting with directory: {f1=D:\flumedev\source\test2\asdfile.log, f2=D:\flumedev\source\test1\test.log}
17 七月 2024 00:00:04,724 INFO [lifecycleSupervisor-1-3] (org.apache.flume.source.taildir.ReliableTaildirEventReader.:84) - taildirCache: [{filegroup='f1', filePattern='D:\flumedev\source\test2\asdfile.log', cached=true}, {filegroup='f2', filePattern='D:\flumedev\source\test1\test.log', cached=true}]
17 七月 2024 00:00:04,724 INFO [lifecycleSupervisor-1-3] (org.apache.flume.source.taildir.ReliableTaildirEventReader.:85) - headerTable: {}
17 七月 2024 00:00:04,724 ERROR [lifecycleSupervisor-1-3] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:251) - Unable to start PollableSourceRunner: { source:Taildir source: { positionFile: C:/Users/liu970/.flume/taildir_position.json, skipToEnd: false, byteOffsetHeader: false, idleTimeout: 120000, writePosInterval: 3000 } counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.UnsupportedOperationException: View 'unix' not available
at sun.nio.fs.AbstractFileSystemProvider.readAttributes(AbstractFileSystemProvider.java:91) ~[?:1.8.0_43]
at java.nio.file.Files.readAttributes(Files.java:1964) ~[?:1.8.0_43]
at java.nio.file.Files.getAttribute(Files.java:1869) ~[?:1.8.0_43]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.getInode(ReliableTaildirEventReader.java:284) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.updateTailFiles(ReliableTaildirEventReader.java:248) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.(ReliableTaildirEventReader.java:93) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.(ReliableTaildirEventReader.java:49) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader$Builder.build(ReliableTaildirEventReader.java:355) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.TaildirSource.start(TaildirSource.java:105) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:71) ~[flume-ng-core-1.11.0.jar:1.11.0]
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249) ~[flume-ng-core-1.11.0.jar:1.11.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_43]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_43]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_43]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_43]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_43]
17 七月 2024 00:00:07,728 INFO [lifecycleSupervisor-1-8] (org.apache.flume.source.taildir.TaildirSource.start:94) - r1 TaildirSource source starting with directory: {f1=D:\flumedev\source\test2\asdfile.log, f2=D:\flumedev\source\test1\test.log}
17 七月 2024 00:00:07,728 INFO [lifecycleSupervisor-1-8] (org.apache.flume.source.taildir.ReliableTaildirEventReader.:84) - taildirCache: [{filegroup='f1', filePattern='D:\flumedev\source\test2\asdfile.log', cached=true}, {filegroup='f2', filePattern='D:\flumedev\source\test1\test.log', cached=true}]
17 七月 2024 00:00:07,728 INFO [lifecycleSupervisor-1-8] (org.apache.flume.source.taildir.ReliableTaildirEventReader.:85) - headerTable: {}
17 七月 2024 00:00:07,728 ERROR [lifecycleSupervisor-1-8] (org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run:251) - Unable to start PollableSourceRunner: { source:Taildir source: { positionFile: C:/Users/liu970/.flume/taildir_position.json, skipToEnd: false, byteOffsetHeader: false, idleTimeout: 120000, writePosInterval: 3000 } counterGroup:{ name:null counters:{} } } - Exception follows.
java.lang.UnsupportedOperationException: View 'unix' not available
at sun.nio.fs.AbstractFileSystemProvider.readAttributes(AbstractFileSystemProvider.java:91) ~[?:1.8.0_43]
at java.nio.file.Files.readAttributes(Files.java:1964) ~[?:1.8.0_43]
at java.nio.file.Files.getAttribute(Files.java:1869) ~[?:1.8.0_43]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.getInode(ReliableTaildirEventReader.java:284) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.updateTailFiles(ReliableTaildirEventReader.java:248) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.(ReliableTaildirEventReader.java:93) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader.(ReliableTaildirEventReader.java:49) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.ReliableTaildirEventReader$Builder.build(ReliableTaildirEventReader.java:355) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.taildir.TaildirSource.start(TaildirSource.java:105) ~[flume-taildir-source-1.11.0.jar:1.11.0]
at org.apache.flume.source.PollableSourceRunner.start(PollableSourceRunner.java:71) ~[flume-ng-core-1.11.0.jar:1.11.0]
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249) ~[flume-ng-core-1.11.0.jar:1.11.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_43]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[?:1.8.0_43]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_43]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_43]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_43]
at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_43]