How should upper-layer applications handle permanent write errors and extent cleanup?
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 35/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Stack tecnológico
- c
- Área
- backend, operating-systems
Línea de trabajo
Comienza rastreando el comportamiento visible para la aplicación de write(), fsync(), close() y ltfs.sync() en relación con las entradas de registro de error de escritura permanente y de limpieza de extents del issue. Se considera terminado cuando estén definidas la semántica de las escrituras correctas, la limpieza, la detección de errores y el manejo recomendado después de que ltfs.sync() devuelva -1, incluido cómo distinguir las causas del dispositivo, del medio y de la ruta de I/O.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
I performed a test in which I wrote four files to a tape using four parallel threads. The four threads use a lock around the write() system call to ensure that the drive is accessed serially. However, operations such as open(), fsync(), and close() for each file are not protected by the lock.
After all four jobs completed, I called ltfs.sync() once.
During the writing of the second file, a "Cannot write block" error occurred. The tape entered a permanent write error state, and LTFS immediately performed an extent cleanup. The relevant LTFS log is as follows:
Sep 15 01:54:53
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 64 LTFS11337I Update index-dirty
flag (1) - NO_BARCODE (0x0x55c5635ea710).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS12045E Cannot write block:
backend call failed (-20301). Dropping to read-only mode.
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS11072E Cannot write blocks:
failed to write to the medium (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 65 LTFS11077E Cannot write: failed
to write blocks to the medium (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.warn ltfs[98]: 65 LTFS13014W Data partition
writer: failed to write data to the tape (-20301).
Sep 15 01:55:29
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS13024I Clean up extents and
append index at index partition (-20301).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17292I Current position is
(1, 10246), Error position is (1, 10117).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS13025I Truncate extents
larger than position (1, 10117), block size = 524288.
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11334I Remove extent :
utdm.0.0.1613966600.2147720233 (9407, 439877632).
Sep 15 01:55:31
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11343I Try to write an index
on the IP on NO_BARCODE because of a permanent write error on the DP..
Sep 15 01:55:49
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17235I Writing index of
NO_BARCODE to a (Reason: Write perm, 5 files) 11C9D0D0D3.
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS17236I Wrote index of
NO_BARCODE (Gen = 2, Part = a, Pos = 8, 11C9D0D0D3).
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.info ltfs[98]: 65 LTFS11337I Update index-dirty
flag (0) - NO_BARCODE (0x0x55c5635ea710).
Sep 15 01:55:58
vdm3000-udm-utdmagent-0 user.err ltfs[98]: 8c LTFS14022E Cannot flush file data
to the medium. Data might be lost (ltfs_fuse_flush).
On the application side, I observed that the write() calls for the second, third, and fourth files returned errno = 2. Then ltfs.sync() returned -1.
In this situation, LTFS performed an extent cleanup after the permanent write error. My questions are:
-
Can the contents of the first file, for which some
write()calls had already returned success, be partially or completely affected by the cleanup?In other words, is it possible that some data from the first file that had previously been reported as successfully written by
write()was truncated or removed because of the subsequent permanent write error? -
How can an upper-layer application reliably detect that the tape has entered the permanent write error ("write-perm") state?
Is there an LTFS API, ioctl, status interface, return code, or other mechanism that an application can use to detect this state without parsing the LTFS log?
-
How can the upper-layer application determine that LTFS has performed an extent cleanup or rollback?
Is there any API, event, status, or other mechanism that exposes this information to the application?
-
What should the upper-layer application do after
ltfs.sync()returns-1?What checks are recommended to determine whether previously successful writes are still valid and whether any data may have been lost or truncated?
-
More generally, what guarantees does LTFS provide for a successful
write()in the presence of a subsequent permanent write error and extent cleanup? -
It is quite possible that the "Cannot write block" error was caused by a drive-related problem. In that case, how can we reliably determine whether the root cause was the drive itself, rather than the tape medium or the I/O path (e.g., HBA/FC/backend)?
I would like to understand the expected semantics in this situation and the recommended way for an upper-layer application to detect and safely handle a permanent write error and the resulting extent cleanup.
- Lenguaje dominante
- C
- Estrellas
- 358
- Forks
- 111
- Merge medio
- 5 d 1 h
- PR fusionados (30 d)
- 2
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de LinearTapeFileSystem/ltfs
-
2.4.9.1 mkltfs failedPosiblemente ocupada @madjesc la tomó hoy. AbiertoInvestigating
LinearTapeFileSystem/ltfs#656 · 1 comentario · 1 asignado ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 25/100
LinearTapeFileSystem/ltfs#650 ·
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
LinearTapeFileSystem/ltfs#643 ·
-
Format fails on HP LTO-6 6250 with Adaptec HBAQuizá libre de nuevo @madjesc la tomó hace 44 días y no hay ningún pull request abierto. AbiertoH/W issue HBA Report Investigating
LinearTapeFileSystem/ltfs#640 · 8 comentarios · 1 asignado ·
-
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
LinearTapeFileSystem/ltfs#638 · 2 comentarios ·
Todos los issues de LinearTapeFileSystem/ltfs
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
fluent/fluent-bit#12579 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
obsproject/obs-studio#14036 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 72/100
-
Feature Request
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
sandboxie-plus/Sandboxie#5653 ·
Los mantenedores suelen responder en 1 día