Yvonne Tunnat

Motivation

As most of us rely on JHOVE for file format validation, it is important to gain information about the quality of JHOVE’s validation skills – and, if inadequacies are detected, improve JHOVE. This can be done by tool benchmarking, as I already did with JPEG in 2016 and with TIFF earlier this year and as Johan did with the WAVE format.

Our Digital Archive currently holds almost 50.000 GIF files and we use JHOVE to validate them.

I have a sample of more than 48,000 GIF-files for this examination. 221 of these files are interesting in one of the following ways:

1. Either JHOVE or one of the other tested tools flagged the file as invalid
2. A self-built java-tool revealed that the file header was not correct
3. A self-built java-tool revealed that the trailer was not correct

All the rest were just valid and not flagged by any of the tools as invalid, so I left those files out of the statistics within this blogpost.

Initially, I worked with only 1,000 files and could also check if there was something visibly wrong with the file and if it could be opened with one of the three following viewers: Internet Explorer, ImageMagick or Paint. This was not possible with the 48,000+ files, but I checked if there was a preview thumbnail for the image in the folder. Besides, I checked renderability for the 221 “interesting” files.

JHOVE and GIF

Graphic Interchange Format (GIF) does not seem to be that widespread in Digital Preservation. There is a JHOVE module which is able to validate GIF files, however, the GIF module is among the smaller of all the modules, but this might just be due to the simplicity of the GIF format.

GIF header & GIF trailer

Depending on the GIF flavor, a GIF file always has to start with one of the two following ASCII strings:

• GIF87 (released 1987)
• GIF89a (released 1989)

To be conform with its file format specification, a GIF always has to end with the HEX “3B”. This is how a valid trailer looks like:

An invalid trailer might look like this:

A common error here is that some of the sample files have not been fully uploaded or downloaded and there is something missing at the end. With complex file formats like PDF this usually leads to files which cannot be rendered any more. With the simpler GIF format, the files can still be opened, but often there was a visibly missing chunk:

 Examination – Findings of the four tools

For my tests, I used GIF files from different sources, e. g. OPF govdocs, files from co-workers and from our own archive. The validity was checked with four tools:

JHOVE (version 1.14.6)

ImageMagick (version 7.0.3-Q16)

NLNZ Metadata Extractor (version 3.6GA)

Bad Peggy (version 2.2)

Of these four tools, only JHOVE and ImageMagick are really meant for GIF validation.

The Metadata Extractor is not for validation, but for metadata extraction. But a side effect is that it throws errors if a file is too broken to extract metadata properly.

Although Bad Peggy is mainly intended for JPEG validation, it also understands GIF (and png and bmp) images, as far as the java image IO understands these. The validation of GIF is not as extensive as for JPEG files and pretty much rudimental, but it was worth a try.

Within this blogpost, a file within the sample is considered valid, if there is nothing visibly wrong with the file and header and trailer are ok. I am aware that it could disobey  its file format specification in another way, which is more difficult for me to detect, at least for now.

All files that either

1. can’t be opened
2. are visibly missing a chunk
3. have an incorrect trailer
4. have an incorrect header

are considered invalid within this blogpost.

So, how do I determine the quality of a validation tool? It depends on how I use it, of course.

Scenario 1: using a tool as the only validation tool there is in the Preservation system

In this case I do not want the tool to miss any invalid file. There should not be any false negatives (=invalid files that are not detected by the tool).

Well, in this case: Hands away from Bad Peggy! It misses 188 files, and considering the pool of “interesting” files in this sample is 221, this is a very bad job. As I have discussed in an earlier blogpost, Bad Peggy is best suited for JPEG validation. GIF validation does not seem to be the tool’s strength. Given that the name is “Bad Peggy” rather than “Bad Giffy”, this is not surprising.

I have included the “bad trailer” in this grapfic, as I wanted to test if all the files which cannot be rendered lack a valid trailer as well. Most of them do. However, 10 of them have a perfectly fine trailer, but either cannot be rendered at all or there is a visibly missing chunk of the image when it’s displayed.

JHOVE and ImageMagick are evenly matched here. In fact, they miss problems in almost the same files within the corpus. There are two files that JHOVE catches and ImageMagick misses (testfiles 30350.gif and 30400.gif) and one file that ImageMagick catches, but JHOVE misses (testfile 16533.gif). As this examination should serve to improve JHOVE’s validation skills, I have created a GitHub issue to document this case.

Scenario 2: not much staff

In this scenario, false positives (false alarms), files that are (or might be) perfectly valid, but are flagged as invalid by the tools are the thing I dislike most, as they cause me unnecessary work.

Well, JHOVE is not my favourite tool then, and neither is the Metadata Extractor. ImageMagick, meanwhile, rings the false alarm only once.

Scenario 3: I have the heart of a scientist and want the tools to just work and deliver correct findings

Both false negatives and false positives are unwanted in this scenario, I just want as many correct findings as possible.

Interestingly, the Metadata Extractor clearly wins here. Having no false negatives at all, even the 18 false positives cannot do much harm to the good output of the tool – only it is not really meant for validation.

ImageMagick still does a lot better than JHOVE, having only 1 false positive compared to JHOVE’s 16 false negatives.

Conclusion

JHOVE does not do so bad, but does not win the contest (yet), either. Of course, there is the possibility that the 16 presumably false positive files are indeed disregarding the file format specification in some way – in a way that does not affect the header, the trailer or the renderability in contemporary viewers, but might cause problems in the near future and bear a risk for long-term-availability.

The next step for me will be to build GIF files from scratch, mess with the structure and then test if JHOVE detects the error or not. This file test suite can then be used as a ground truth set and might be included in GitHub to enrich the regression testing for further JHOVE versions.

All findings can be found on google docs. As some of the files cannot be openly published, I cannot publish the GIF files. But I am willing to share interesting files via Email, just get in touch: y.tunnat@zbw.eu

The research question

I have never doubted the JHOVE TIFF module. The JHOVE TIFF module is always right. Everybody says so. That’s why nobody uses the myriad alternatives to it, although it’s so easy to write a TIFF validator, I could almost do it myself.

But while my colleague Michelle and I are drafting a paper for the IDCC this february, it dawned on me: “Everybody” has never written about the infallibility of JHOVE in a paper or Blogpost so far, or has run a thorough test that I know of. Besides, the “myriad alternatives” often seem difficult to use for me on my windows machine with my limited experience with command-line-tools and batch-scripting.

Last fall, I have compared the validation tools JHOVE and Bad Peggy and how they both deal with JPEG validation (see OPF Blogpost). My goal was to analyse if the JHOVE JPEG module is reliable, as we are basing our preservation decisions on it. In theory, my goal is the same with this examination, except being focussed on TIFF, but my initial intention was admittedly biased: I wanted to prove that the JHOVE TIFF module indeed is infallible and that TIFF validation is, as I have always known, easy-peasy. As the analysis went on I had to admit that the reality is much more complicated.

The statement of a validation tool usually is relied on without a second thought, although most validation tools are not free from false negatives and false positives. As the JHOVE validation tool is widespread in the digital preservation community and integrated in out-of-the-box digital preservation software like Rosetta and Preservica, the reliance of JHOVE is especially interesting for the formats we possibly all have in our archives, like TIFF images.

My research question: Is the JHOVE TIFF module really that good in comparison with other tools?

And, as a side-effect: Is TIFF validation really easy-peasy?

TIFF validation tools

First, there seem to be plethora of tools to test TIFF-validity, analyse TIFF-tags and even repair common errors. Some are listed in COPTR (search for “TIFF” and “validation”, though some tools like ExifTool do some validation and are not marked as validation tools). Furthermore, the libtiff library offers many programs that can be integrated in other tools (see e. g. this collection of TIFF-tools). Most tools are not out-of-the-box tools with a nice GUI like JHOVE, which can be used by you and me on a windows machine.

I selected the following tools for my test:

	Validation Tool	version	How to use	remark
1	JHOVE	1.14.6	GUI and java library
2	ImageMagick	7.0.3	Command-line, batch-script	help for the batch-script via twitter from David Underdown and the ImageMagick people
3	ExifTool	10.37	Command-line, batch-script	help for the batch-script from Mario from the German nestor format identification group
4	DPF Manager	3.1	GUI
5	checkit_tiff	0.2.0	runs on linux only yet	Andreas, checkit_tiff developer from the SLUB Dresden has run the test suite for me
6	LibTIFF	4.0.7	runs on linux only	Heinz from the German nestor format identification group has run the test suite for me

In summary, I was able to analyse the test suite with six tools. I had some help with a suitable batch-script for ImageMagick and ExifTool, but at least I could easily run the tests on my windows machine (unlike checkit_tiff, for example, which requires Linux). For checkit_tiff and LibTIFF two colleagues helped me out and sent me the findings for me to analyse.

In the following paragraphs, I introduce the tools I have used for this analysis in more detail:

JHOVE

Validation: JHOVE is my to-go-validator for TIFF files. The findings are intelligible and the expectations for the file to follow the TIFF specification seem reasonable. There are almost 70 known TIFF error and info messages in the JHOVE module, most of them carry their meaning within the message like “TileLength not defined” and even a passer-by with a minimum of fantasy can imagine why information about the tile length might come in handy for an image. In forms of transparency, it is described on the JHOVE website which requirements a TIFF file must met to be well-formed or well-formed and valid.

Handling: As the GUI output never suited me, I have long ago begun to use JHOVE as a java library and have my own html output, which is very user-friendly. Aside from the GUI output not being handy when dealing with many files, JHOVE is very easy to install and use, as one could just throw (drag & drop) files and folders at it and it will validate them all.

ImageMagick

Validation: Just to be fair, ImageMagick is not primarily about file validation, but instead about displaying, migrating and working on images. I have marked every file as invalid that ImageMagick had at least one error message about, even if the error seems to be a minor one, like the encounter of an unknown TIFF field or incorrect contents of tags. As far as I know there is no list of all possible error messages of ImageMagick. The corpus used for this blogpost, the Google Imagetestsuite, however, consists of more than 40 different error messages, listed here. Analysing the ImageMagick output for this Blogpost, “valid” means “error-free”.

Handling: If I would have known how bad the output is before I started this test, I would have skipped this tool altogether. ImageMagick is a command-line-tool and as far as I know the batch-processing only has a text output and this output is really a mess. It’s difficult to tell which error information belongs to which file, as some files are not even listed by name. I had to test those all one-by-one, which was time-consuming and boring. But I am pigheaded and had already started to tell everybody I am going to test this tool. Even after I had written some java to help me with the messy output, some stuff could not be automated, as I could not find a regular pattern for everything. I am very sure that ImageMagick is very useful for batch-processing when converting images etc., but obviously nobody has really thought about validating 166 images at once or validation in the first place.

ExifTool

Validation: ExifTool is not really meant for validation, either. It’s for metadata extraction. The information about image errors is just a by-product if the tool runs into any problems while trying to extract metadata. So it’s not really fair to treat ExifTool like a validation tool, as it would never complain about an absolute unreadable TIFF which cannot be opened by any viewer, as long as all the metadata can get extracted. That might be the reason why ExifTool has the highest percentage of presumably valid TIFF files within this test. So, “valid” for ExifTool means, that there were no warnings or errors in the metadata output.

Handling: It’s a command-line-tool with quite good possibilities to batch whole folders and output human-readable csv (though the csv can have many, many columns, as images can have a myriad of metadata).

DPF Manager

Validation: The DPF Manager is for TIFF validation only and was built only for that. Some validation profiles are included (e. g. for “baseline TIFF”, “extended TIFF”, TIFF/EP and the TI/A Draft). Besides, you can specify your own validation profile to check against. For this analysis, only the baseline and extended TIFF profiles were used. For the DPF manager, “valid” means that a TIFF file does not hurt the specification in any way (depending on the profile used).

Handling: The DPF Manager is very easy to install (though you need Admin-rights to do so, it’s not portable like JHOVE is) and extremely easy to use: You can just drop and drag a file or folder on the GUI or, alternatively, select a file or folder. The tool is very fast – 80 TIFFs need less than a minute (size of the TIFFs varied between 9 kb and 4 MB) – and the HTML output is very nice and there is also METS and XML (although personally I would like an additional csv option). Furthermore, the TIFF files are sorted by the number of errors. The worst ones come first and when you scroll down, the TIFF get less and less invalid, ending with the valid ones in the end. So bad news first! There are also thumbnails of the TIFFs, if the image is renderable. I think in terms of usability this tool has easily won the contest.

As a bonus, each error is referenced to the page and section in the TIFF guide, including the exact quotation that the error refers to.

Having seen this, I am tempted to think of the DPF manager as the reference tool for whether a TIFF really is valid or not – the question is whether it outperforms my current go-to tool, JHOVE? It certainly aims at being the go-to-validator for TIFF-files.

checkit_tiff

Validation: checkit_tiff is too picky for this examination, as it validates against baseline TIFF whereas the TIFFs in the Google Imagetestsuite are not baseline. I still want to present the findings of the tool and include it in this Blogpost, as the reader surely has a different TIFF corpus and might very well want to validate baseline TIFF.

Handling: The tool runs on Linux. There is not yet a windows version of the tool and it’s a command line tool. A windows version is about to be released soon, though.

LibTIFF

Validation: There are 695 different error messages from evaluation of the Google Imagetestsuite sample (listed here), mostly very similar ones dealing with unknown TIFF tags. Reading the error messages, LibTIFF seems to check the tags only, and omits some general file-structure validation, such as the check for end-of-file tags. It does check the TIFF header for the magic number, though (see “No TIFF magic number“). As far as I know, there is only a text output (“.log”), which in general is readable, but for bulk-analysis not much better than ImageMagick.

Handling: The tool runs on Linux.

Test corpus

The test was run on the Google Imagetestsuite for TIFF, which has the advantage of being openly available and consists of some really bad TIFF files. By that I mean that half of the files are not even renderable or looks bogus, as if parts were missing or the image is just grey, white or black, and one does not quite know if that’s intentional.

The files are named after their MD5-checksums (see: About).Unfortunately, there is no “ground truth” indicating whether each TIFF file is valid or not. I have added information about whether the image is renderable in either Windows Photo Preview, Paint or ImageMagick in the Findings spreadsheet, and used this as a basis for comparing tool validation output against. I know that this is not a water-tight solution as an image can be invalid and still open in a current viewer. Some of the Google Images also look damaged in the viewer: either as if something is missing or just black or white – and I cannot figure out if this is on purpose (= this just is a picture of some black stuff) or the image is broken.

Reading the explanation of the TIFF Google Imagetestsuite, all images without the prefix “m-” are original and were not modified. All images with the prefix “m-“, however, were modified in some way. Although the intention was not necessarily to add errors to the image, the percentage of valid images for these files is much smaller (see table below).

For the chapter “differences in terms of error” below I have used the 3 files of the fixit/checkit_tiff test corpus in addition, as these files were better suited for the examples.

Examination of 166 TIFFs from the Google Imagetestsuite

For ImageMagick and ExifTool I have marked images as invalid if they threw errors.

If an image could not be analysed by a tool at all (e. g. because the tool only analysis files with a correct TIFF header and some files don’t provide one), they were marked as “could not be analysed”.

I have tested the images wicht windows photos, paint, ImageMagick and the thumbnail preview on DPF Manager. If an image could not be rendered by either of them, I marked it as “invalid” in the column “renderable in a viewer”.

A few observations on the above figures:

checkit_tiff does not analyse the TIFF files if the magic number is missing.

Compared against the ability for a viewer to render the files, all tools seem to generate false positive (=false alarm) results – more files are marked as invalid than renderability would seem to imply.

The number of invalid “modified files”, and the number of invalid “non-renderable files” is very high. There were some files which could not be opened with Paint but with ImageMagick, though (marked with “ImageMagick can open” in the Spreadsheet).

I am reluctant to state that JHOVE has two false negatives here, but all the other tools have marked these two files as invalid except ExifTool, which has marked one of the files as error-free (see Spreadsheet). Furthermore, no viewer can render the files. I tried to analyse the files with my own TIFF java tools, but they could not process the files at all. So much as I hate it, I have to admit: These are false negatives. What else should I call it? I certainly would not want these two files going unnoticed in my archive. Furthermore, the other five tools all have detected that something is wrong with these files.

Premature End-of-File

For 5 files of the test corpus JHOVE throws the “Premature End-of-File“-Error, which usually hints at a fatal error with the file. Often parts of the file are missing, a typical issue is that the file was not completely downloaded/uploaded and the last chunk of the file is not there. JHOVE usually realises missing chunks at the end of a file.

Four of the five files (spreadsheet) do look very suspicious. Two cannot be opened, two are black, one looks as if parts of the text were missing. At least most tools agree that something is wrong with the files. Only the DPF manager considers one of the 5 files to be valid. Looking at the error messages of the DPF manager, there is no hint of a premature file ending.

ImageMagick throws the error “unexpected end-of-file”, but for five other files of the corpus (listed here). JHOVE reports other errors for these files, but at least again all validation tools agree on the invalidity of the files. They certainly look damaged in the viewer and one of them cannot even be opened. The DPF manager could not even analyse these five files, they all were omitted in the analysis.

No TIFF magic number

If the file only purports to be a TIFF file, e. g. by the file extension, but the magic number cannot be found, all tools agree on the error (spreadsheet). None of the three files reporting the error could be opened and JHOVE, ImageMagick, DPF Manager and checkit_tiff (by not handling the file) agree that the magic number is missing and that it therefore cannot be a TIFF file or rather the TIFF signature is incorrect.

Commonalities in terms of errors

Sometimes, the tools agree on an error and even use very similar words to describe the error. One example is shown in the table below.

Obviously, both tools check for unknown TIFF tags and reports it if they encounter some. ImageMagick also gives the Hex value of the field. It does not matter which of these two tools one uses, it will always report unknown tags. At least both tools have done so with the Google Imagetestsuite.

Differences in terms of errors

For theese two examples I have used files from the fixit/checkit_tiff test corpus.

JHOVE: “Invalid DateTime separator”

The JHOVE Module reports correctly if the DateTime is not formatted as it supposed to be and marks the file as “well-formed, but not valid”. It has done so with the “invalid_date.tiff” from the fixit / checkit_tiff testfiles. ImageMagick, however, completely neglects to realise that there is something wrong with the DateTime Tag in this file and the error goes unnoticed. (ImageMagick does report an error, which seems to be unconnected to the DateTime, however, as it is about the “Photoshop”-tag.) The DPF manager also reports “Incorrect format for DateTime” and quoted the TIFF specification, so this is a false negative for ImageMagick.

JHOVE: “Value offset not word-aligned”

The JHOVE module throws this error for the “minimal_valid”-Tiff in the checkit_tiff Examples and marks the TIFF as “not well-formed”. ImageMagick does not report any errors for this file. The DPF Manager, however, finds three errors in the file, two related to “bad word alignment in offset” (which sounds pretty much like the JHOVE error) and one inconsistency about the tag planar configuration, which does not sound that fatal (“PlanarConfiguration is irrelevant if SamplesPerPixel is 1, and need not be included.“).

Fun fact

Of the 166 files, only for four files do all of the tools (except checkit_tiff, which considers them all to be invalid) agree on validity (spreadsheet). If one would decide on a file validity policy which only allows files in an archive for which no tools has any complaints, it would be a very empty archive indeed. It might not even be possible to satisfy them all with real-world images from different producers.

Summary and conclusion

Although the tools agree on the “real bad” TIFF files, TIFF validation does not seem to be at all that easy-peasy. It has been much easier – at least with the corpus analysed – to determine what is a false positive and what is a false negative with the JPEGs in my last OPF Blogpost. The JHOVE TIFF module still seems to be a decent choice and I have not found any real gap like I did with the JHOVE JPEG module the other day, although the two false negatives leave me nervous.

Findings of the DPF manager seem to be trustworthy to me, as the TIFF specification can be referenced for each error found. Please note that some errors lead the DPF manager not to detect TIFF files as such, e. g. if the file ends prematurely or unexpectedly (see this spreadsheet).

Nevertheless, most of the tools – if not all – seem to be too paranoid. Assuming all non-modified TIFF are valid (which are all renderable in a viewer), only ExifTool considers 94% of them to be valid (or, “error-free”, as in the case of ExifTool). The second-best, JHOVE, still considers almost half of them to be invalid in some way. The DPF manager considers only 30% of them to be valid (Extended TIFF) and even is able to prove every bit of it.

Back to my research questions:

Is the JHOVE TIFF module really that good in comparison with other tools?

Well. It’s pretty user-friendly, the error messages are intelligible (but most TIFF errors are, with every tool tested), the output can be dealt with, but it’s not as user-friendly as the DPF manager, which also has a nicer output. And, the DPF manager has the reference to the specification all the time, which really feels good when talking to my boss about the quality of our TIFF files. Look, the TIFF bible says it’s ok / not ok. Who would argue?

Nevertheless, it was the only (real validation) tool with false negatives with perfectly invalid and un-renderable files, which would be worth a second look in one of my next posts.

And, as a side-effect: Is TIFF validation really easy-peasy?

It does not seem so, as the validators agree on very little indeed.

So, how to act?

I might just stick to JHOVE in our productive digital preservation environment, but I will at least add the DPF manager in our Pre-Ingest workflows, especially in our digitisation centre, to be sure we stick to the TIFF specification at least with TIFF files we generate ourselves. When receiving files from outsiders, I will be more tolerant, as I always am, but might add a preservation planning workflow to repair the TIFFs, if possible. But that will be the topic of another post at another time.

In this Blogpost I want to examine the findings of two validation tools, JHOVE (Version 1.14.6) and Bad Peggy (version 2.0), which scans image files for damages, using the Java Image IO library. Goal is to compare the findings and enable the reader to know what to expect from these validation tools for the daily digital curation work.

This test was done with the publicly available Imagetestsuite from Google. Furthermore, I included pictures from events like Christmas parties and outdoor events in my library from the last 6 years, pictures contributed by friends and colleagues, some of my own pictures, and even some memes from public fun Facebook pages like “useless facts”.

In one case, I even opened a JPEG in an editor to remove some bytes to test if the tools would realise that something was wrong, hoping I would get error messages that were still missing from the list (which, by the way, worked out).

All in all 3070 JPEG files were tested. In the following, I will focus on the files that either the JHOVE JPEG module or Bad Peggy – or both – found objectionable in one way or the other.

In general, the JHOVE JPEG module knows 13 different error messages, whereas Bad Peggy can distinguish at least 30 (source code of KOST-Val, which uses Bad Peggy to validate JPEG files).

Table: Possible JHOVE errors on JPEG files

In this table there are listed the JHOVE errors only. All errors found by Bad Peggy in the sample are in the table below the conclusion.

	Message	Examples in the sample?	No. of occurences in Sample	Bad Peggy equivalent
1	DTT segment without previous DTI	no	/
2	Unexpected end of file	Yes (Example)	26	corrupt data: premature end of data segment. AND corrupt data: Truncated File – Missing EOI marker.
3	I/O exception processing Exif metadata:	no	/
4	Invalid JPEG header	Yes (Example)	4	The file is not a JPEG (header).
5	JFIF APP0 marker not at beginning of file	Yes (see “Expected marker byte 255, got”)	17	Bad Peggy does not recognise this file as invalid
6	Marker not valid in context	Yes (Example)	3	invalid file structure: two SOI markers
7	Expected marker byte 255, got	Yes (Example)	78	Bad Peggy does not recognise this file as invalid
8	SPIFF marker not at beginning of file	no	/
9	File does not begin with SPIFF, Exif or JFIF segment	Yes (see “Expected marker byte 255, got”)	105	Bad Peggy does not recognise this file as invalid
10	Error creating temporary file. Check your configuration:	no	/
11	Unrecognized tiling data	no	/
12	Value offset not word-aligned: xxx	Yes (Example)	6	Bad Peggy considers these files to be invalid as well, but throws different error messages
13	No TIFF magic number: 4906 (Is officially a TIFF error message, but was thrown for this presumably JPEG file)	Yes (Example)	1	corrupt data: bad Huffman code. (only one file in the sample)

So for five of the error messages no examples could be found, so I won’t look at these errors in this blog post. Furthermore, for the error “No TIFF magic number: 4906” and “Value offset not word-aligned: xxx“, the examples in the sample were so scarce that I cannot possibly explain them properly yet. If anybody out there has examples for these errors, I will happily extend this post and include the findings.

Let’s take a closer look at the different errors:

General information about the JPEG structure

Between the SOI (Start of Image) and the EOI (End of Image), there are other segments allowed, roughly said the structure of a JPEG should be as following:

• SOI-segment (SOI: start of image): “FF D8”
• APP0-segment (JFIF-Tag): “FF E0”
• other segments
• SOS-segment (SOS: start of scan): “FF DA”
• data: compressed data
• EOI-segment (EOI: end of image): “FF D9”

If there are e. g. embedded thumbnails, it is also possible that a single file can have more than one SOI- and EOI-segments. In these cases, the JPEG has to have a SOI segment between two EOI thumbnail markers. (Bad Peggy checks this, whereas JHOVE does not.)

Unexpected end of file

Interestingly, JHOVE does not always detect if parts of the files are missing. Only for files where Bad Peggy throws two errors: “corrupt data: premature end of data segment” AND “corrupt data: Truncated File – Missing EOI marker“, will the JHOVE JPEG module detect that something is indeed wrong with the file. For several files Bad Peggy only throws “corrupt data: premature end of data segment” and JHOVE considers them to be valid.

Furthermore, the Bad Peggy error “corrupt data: xxxx extraneous bytes before marker 0xd9.” goes unnoticed by JHOVE. Images in this sample with these errors, though, do not look healthy to me at all.

Jhove detects that there is something wrong with these images (“Unexpected end of file“):

JHOVE does not detect that these images are damages, though Bad Peggy does (“corrupt data: premature end of data segment“):

There is clearly something missing, as you can see with these three examples, which are considered to be valid by JHOVE:

As a digital archivist, I would want to know about these errors while ingesting data. I fully agree with Bad Peggy – this data is indeed corrupt. I consider this as a false negative finding of the JHOVE JPEG module: The JPEG has serious problems, but JHOVE does not detect them. In this case, Bad Peggy is the better Sherlock Holmes.

The last Bytes of a JPEG should look like this and always end with and EOI (end of image), which is “FF D9”:

In this example, the JPEG just ends without the necesary EOI:

If I add the “FF D9” manually and save the file, JHOVE does not detect that something is wrong with it any more. It considers the file to be valid & well-formed.

Bad Peggy, however, still considers this file to be invalid:

Invalid JPEG header

Easy and straightforward: Both tools check the JPEG header and throw an error if there is no correct JPEG header. This is extremely useful, as tools usually cannot open files with a missing JPEG header. In most cases the file is unreadable for good – or it’s not even a JPEG in the first place.

JFIF APP0 marker not at beginning of file

After the SOI (“FF D8”), an APP0-segment should follow, which always starts with “FF E0” (see: “General information about the JPEG structure”). If this error is thrown, the APP0-segment does not follow directly after the SOI-segment. A JPEG file which throws this error viewed in a Hex editor shows that the  SOI marker “FF D8” is not followed by “FF E0”. Instead, in this example the SOI-marker is directly followed by a copyright marker (“FF EE”).

JHOVE flags this file as missing the APP0 marker; Bad Peggy, however, completely ignores this error and obviously does not test it. The JPEG standard clearly states that JPEG files have to be structured like this, but so far, none of the JPEG files of the sample have caused any problems in commonly used viewers. This cannot be marked as a false positive for the JHOVE JPEG module, but currently does not seem to bear any practical risks for the affected data. As usual, the viewers are more flexible than the file format standard, at least contemporary viewers (we cannot guess for future viewer, however, which makes the decision for long-term-availability harder).

Marker not valid in context

The examples in the corpus are all visibly corrupted, as parts of the images seem to be missing:

The JHOVE error sounds pretty general. For the three examples in the corpus, Bad Peggy has found different equivalents.

Expected marker byte 255, got xxx

This error occurs several times within the sample and gives a plethora of marker bytes which have been used instead of 255. So far, none of the affected JPEGs have shown any problems and Bad Peggy ignores the error altogether.

File does not begin with SPIFF, Exif or JFIF segment

A JPEG file usually uses the graphic format JPEG Interchange Format (JFIF), but can also use Exif or SPIFF – but obviously has to start with one of these three segments and no other. Bad Peggy marks these files as invalid as well, but the error message is quite tight-lipped “ype.” – which was translated by the KOST (Switzerland) as something like “This JPEG contains characteristics that are not supported” – which does not really enlighten me.

JHOVE usually detects this error combined with “JFIF APP0 marker not at beginning of file” or “Expected marker byte 255, got XXX“. An example for a file with this error “standalone” is this:

Similarly to the example for the JHOVE error “JFIF APP0 marker not at beginning of file“, the tag directly following the SOI (“FF D8”)-marker is not a JFIF, Exif- or SPIFF-marker, but a copyright tag (“FF EE”).

Bad Peggy: Invalid file structure: Missing SOI between two EOI thumbnail markers.

Bad Peggy also detects an error that is completely ignored by the JHOVE JPEG module, which Bad Peggy has found for more than 100 files within the sample. This error is almost self-explanatory, knowing what an SOI (start of image) and EOI (end of image) is. So far, none of the JPEGs look bogus in any way or had any problems to be displayed.

Conclusion

After a closer look at the affected JPEG data I would not want these JPEG files being unnoticed in my archive:

Two of the images cannot even be opened and displayed any more and the rest has missing parts, mixed up parts and colour problems. For practical reasons, I would want my tool to detect the errors automatically and not necessarily more than those. These are the only JPEGs that obviously have problems, others show errors in JHOVE or Bad Peggy or both, but contemporary JPEG viewer tools have no problems displaying the JPEGs. Of course it is impossible to say if future tools will be able to display these JPEGs properly.

Considering this, Bad Peggy has clearly won: It detects all of the visually corrupt images.

The JHOVE JPEG module misses 7 out of 18 – which is the Bad Peggy error “corrupt data: premature end of data segment” without the additional error “corrupt data: Truncated File – Missing EOI marker” and “xxxx extraneous bytes before marker 0xd9.” Maybe JHOVE would be just fine if these two extra tests would be included. If there is seriously other stuff missing – well, maybe we’d need a bigger sample to examine to be able to answer this question.

These seven JPEGs with visible problems are missed by JHOVE:

All errors found by Bad Peggy

All in all 1007 files in the 3070 sample had problems, if Bad Peggy is to believed. As one file can contain more than one error, the findings are as follows: