BUG: CTF writer writes the header and the first data row on one line
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 76/100
Direzione di ricerca
Read the header handling in Source/EbsdLib/IO/HKL/CtfReader.cpp, especially getHeaderLines(), readData(), and writeFile(), then review Source/Test/CtfReaderTest.cpp and the existing TestWriteCtfFile case. Run the proposed WrittenFileReadsBack test to confirm the header retains its line breaks and the written file can be read again; check any reader that depends on H5CtfImporter’s OriginalHeader dataset before changing its stored format.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Problem
CtfReader stores the header of a .ctf file without its line breaks, and CtfReader::writeFile() writes that stored header as it is. The written file has every header line, the column-name line and the first data row joined on its first line. CtfReader cannot read that file back: readFile() throws std::invalid_argument ("stoi: no conversion").
Expected: a file written by writeFile() has the same header lines as the input and reads back with readFile(). Actual: its first line holds the whole header, the column names and the first data row (see the test output below), and reading it throws.
Occurs at 34844ed31dfb8660c67edb6e019ca708e04beeb4 (develop).
Steps to Reproduce
- Read
Data/EbsdTestFiles/Test_US_1.ctfwithCtfReader::readFile(). - Write it with
CtfReader::writeFile()to a new path. - Compare the first line of the written file with the first line of the input (
Channel Text File): the written line holds the whole header and the first data row. - Read the written file with a new
CtfReader:readFile()throws "stoi: no conversion".
Cause
getHeaderLines() reads each header line without its newline and appends it to the original header with nothing in between (CtfReader.cpp#L873-L876, and the phase lines at CtfReader.cpp#L888-L890):
::safeGetline(reader, buf);
// Append the line to the complete header. The trailing newline is already removed at this point
appendOriginalHeader(std::string(buf));
readData() adds the column-name line the same way (CtfReader.cpp#L501-L504):
std::getline(in, buf);
std::string originalHeader = getOriginalHeader();
originalHeader = originalHeader + buf;
setOriginalHeader(originalHeader);
writeFile() then writes the stored header verbatim, followed directly by the data rows (CtfReader.cpp#L1013-L1014):
std::string header = getOriginalHeader();
fwrite(header.c_str(), 1, header.size(), f);
When the written file is read, getHeaderLines() sees one line that contains Phases, splits it on tabs and passes the wrong token to std::stoi. The existing test ebsdlib::CtfReaderTest::TestWriteCtfFile checks only that writeFile() returns 0, so it does not catch this.
Verification Unit Test
File: Source/Test/CtfReaderTest.cpp.
TEST_CASE("ebsdlib::CtfReaderTest::WrittenFileReadsBack", "[EbsdLib][CtfReaderTest]")
{
CtfReader reader;
reader.setFileName(ebsdlib::unit_test::CtfReaderTest::USInputFile1);
REQUIRE(reader.readFile() == 0);
std::stringstream filePath;
filePath << ebsdlib::unit_test::k_TestTempDir << "/CTF_WriteFile_read_back_test.ctf";
REQUIRE(reader.writeFile(filePath.str()) == 0);
// The written header keeps one line per header entry, as in the input file.
std::ifstream input(ebsdlib::unit_test::CtfReaderTest::USInputFile1);
std::ifstream output(filePath.str());
REQUIRE(input.is_open());
REQUIRE(output.is_open());
std::string inputFirstLine;
std::string outputFirstLine;
std::getline(input, inputFirstLine);
std::getline(output, outputFirstLine);
CHECK(outputFirstLine == inputFirstLine);
CtfReader readBack;
readBack.setFileName(filePath.str());
int err = -1;
REQUIRE_NOTHROW(err = readBack.readFile());
REQUIRE(err == 0);
REQUIRE(readBack.getNumberOfElements() == reader.getNumberOfElements());
REQUIRE(readBack.getXCells() == reader.getXCells());
REQUIRE(readBack.getYCells() == reader.getYCells());
}
Command:
ctest --test-dir build/Release -R "^EbsdLib::ebsdlib::CtfReaderTest::WrittenFileReadsBack$" -j 1 --output-on-failure --timeout 1500 --no-tests=error
Output on 34844ed (Release, macOS arm64):
Source/Test/CtfReaderTest.cpp:301: FAILED:
CHECK( outputFirstLine == inputFirstLine )
with expansion:
"Channel Text FilePrj C:\NathB\EBSD processing\701-600-1-f.cprAuthor
[Unknown]JobMode GridXCells 40YCells 5XStep 0.5YStep 0.5AcqE1 0AcqE2 0AcqE3
0Euler angles refer to Sample Coordinate system (CS0)! Mag 200 Coverage 100
Device 0 KV 15 TiltAngle 70 TiltAxis 0Phases 13.231;3.231;5.148 90;90;120
Zirc-alloy4 9 0 0_5.0.6.0 -67395467 [Zr4.cry]
Phase X Y Bands Error Euler1
Euler2 Euler3 MAD BC BS
0 0.0000 0.0000 0 3 0.0000 0.0000 0.0000 0.0000 85 0
"
==
"Channel Text File
"
Source/Test/CtfReaderTest.cpp:306: FAILED:
REQUIRE_NOTHROW( err = readBack.readFile() )
due to unexpected exception with message:
stoi: no conversion
===============================================================================
test cases: 1 | 1 failed
assertions: 6 | 4 passed | 2 failed
Proposed Fix
Keep the line breaks when the original header is stored, so the stored header is the file's header text:
getHeaderLines():appendOriginalHeader(buf + "\n");in both loops. The phase loop reads withstd::getline, so a CRLF file keeps its\rbefore the added\n.readData():originalHeader = originalHeader + buf + "\n";for the column-name line.
writeFile() then writes a header that ends with a line break, and the first data row starts on its own line. The test above then passes. H5CtfImporter also stores getOriginalHeader() as the OriginalHeader dataset of .h5ebsd files, so it would get the original header with its line breaks too. Check any reader of that dataset that depends on the current joined form.
Notes
- Found while writing the verification test for BlueQuartzSoftware/EbsdLib#65 (CTF float precision); that test sets the header with
setOriginalHeader()to get past this defect. - By code reading,
.h5ebsdfiles written byH5CtfImporterstore the same joined header inHeader/OriginalHeader(H5CtfImporter.cpp#L298-L299). This was not tested.
- Lingua principale
- C++
- Stelle
- 8
- Fork
- 16
- Merge medio
- 4h 35m
- PR unite (30g)
- 3
Preparare l'ambiente
Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di BlueQuartzSoftware/EbsdLib
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
BlueQuartzSoftware/EbsdLib#65 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
BlueQuartzSoftware/EbsdLib#55 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 4/5 3-5 giorni Idoneità per principianti 38/100
BlueQuartzSoftware/EbsdLib#47 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 35/100
BlueQuartzSoftware/EbsdLib#45 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 35/100
BlueQuartzSoftware/EbsdLib#35 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di BlueQuartzSoftware/EbsdLib
Issue simili
-
bug triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
I maintainer di solito rispondono entro 1 giorno
-
26.04 bug dark-mode Easy Hack impress symptom: visual
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
CollaboraOnline/online#16421 · 1 commento ·
-
area/cs bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
ydb-platform/ydb#59881 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
I maintainer di solito rispondono entro 1 giorno
-
documentation enhancement lua server
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100