Denise de Vries

CFP: BORN DIGITAL HERITAGE NOW
#BDCH23 – https://aama.net.au/conference/
Thursday, November 30th – Friday, December 1st, 2023

#BDCH23 is the third iteration of the Born Digital Cultural Heritage conference series. The first and second iterations – in 2014 and 2022 – brought together academics and practitioners from across the world to consider issues in the field of born digital heritage, from a range of disciplinary perspectives. A notable feature of this conference series has been the deliberate attempt to establish dialogue between those working in different professional fields. We recognise that born digital heritage reaches into all disciplines and that, if we want to make progress on the challenges of preserving born digital heritage, it takes people from a range of disciplines and professions working together. Therefore, we seek to break down professional silos and other barriers that inhibit the discussion and sharing of questions, knowledge and know how to address common challenges that we all face when researching, preserving and accessing different forms of born digital cultural heritage.

BDCH#23 will consider where born digital cultural heritage is at now. What are we making progress on and what are the urgent challenges? How are we going to safeguard our digital heritage for future generations?

For this event, we aim to expand the fields of discussions to include architecture and design and extend the focus on digital games and media arts to include websites, networking and other relevant domains. The challenges of preserving digital cultural heritage aren’t limited to legacy technologies, of course, but also affect contemporary platforms such as apps, VR, AR, etc.

BDCH#23 will be a hybrid conference held at the Australian Centre for the Moving Image in Melbourne and online via zoom.

Access the full program for the last Born Digital Cultural Heritage Conference held in 2022 with abstracts and recordings of all presentations.

We invite proposals for papers that address the four sub-themes: Collecting, Curating, Preserving and Researching, as well as those that cut across these sub-themes.

Please send an abstract of 300 words, plus keywords and references, and a brief author biography (50-80 words) by July 27th, 2023.

Send proposals to aama@swin.edu.au

An anthology “On Revisiting 1990s Born Digital Artefacts” (working title) is planned after the conference.

Possible topics may include, but are not limited to:

• Recognition of digital cultural heritage
• New approaches to collecting and curation
• Engaging audiences with legacy digital artefacts
• Exhibition case studies
• Policy directions
• Institutional interactions
• Networked initiatives and coalitions
• Distributed collections
• New approaches to loans
• New techniques in preservation
• Imagining future tools, methodologies, problems and/or solutions
• Legal developments
• Tools and practical case studies
• Training and workforce development
• Progress on emulation strategies
• Histories of the born digital
• Reflections from researchers on accessing newly re-accessible artworks
• Collaborations between practitioners and researchers.

The Play It Again team is hosting a free online conference Born Digital Cultural Heritage, 16-18 February 2022, in conjunction with partners
ACMI & AARNet.

Keynotes from Klaus Rechert & Dene Grigar & super panels with international speakers, optimised for speakers’ timezones & recorded.

Conference program & speakers: http://playitagainproject.com/conference
Register at http://ow.ly/Cc2j50HG56w

I’m happy to announce that we have received funding for the preservation  of Australian videogames of the 1990s.

This is a joint research project with Flinders University (South Australia),  RMIT University (Victoria), Swinburne University (Victoria), Australian Centre for the Moving Image, AARNet P/L,  UNESCO PERSIST and OpenSLX GmbH.

This project aims to document, preserve, and exhibit the history of the Australian videogame industry during the 1990s. The challenge of preserving and accessing complex digital cultural heritage such as software is one that collecting institutions worldwide are facing. By recovering the history of Australian made videogames of the 1990s, preserving significant local digital game artefacts currently at risk, and investigating how these can be exhibited as playable software using the latest emulation techniques, the project expects to generate new knowledge needed by government and industry to inform future strategy and infrastructure investment aimed at making a range of digital cultural heritage available to the public.

Games and other software became more complex during the 1990s. Standard computer environments were no longer enough to execute a game. More specialised additional software was required, such as dlls (data link libraries), graphic and sound card drivers. Much digital preservation research to date has concentrated on end products generated by software rather than the software digital object itself. When only the software program is available without the accompanying installation information or files, the computer environment in which the program should best be executed is not easily discovered. Tools need to be developed to discover and document all the requirements for an optimum emulated system that takes into account a multitude of factors (operating system, interface, code, metadata, driver files, etc) needed to emulate a preserved game artefact.

The cultural research team begins from the premise that videogames are not just software or hardware artefacts; digital games are understood as a set of played experiences embedded in wider cultures and communities of use. The 1990s saw the rise of a local studio culture and bold experimentation in the crossover space between cinema and videogames. While focused firmly on digital games, we will attend to the cultural and historical milieu in which game-makers negotiated a changing technology and business scene, collaborating with industries such as film, as well as the player practices around games, including experiments in networked community and what players made and did with these products.

Finally, we have confirmation for the original hardware, an HP9845 with an external floppy drive, the HP9895A Disk Drive attached to it. The disks have 77 tracks, or cylinders, and contain 30 sectors with an interleaving factor of 7. The machine was marketed for scientific purposes and was widely used in the university’s medical centre until the mid 1980s before the IBM PCs were introduced. It offered a wide range of peripheral connections and general programmed input/output (PIO).

Response to our call for help

Responding to a call on cctech for information about decoding a Kryoflux stream for HP M2FM, Eric Smith said that he had already decoded Intel M2FM and most generously added the HP M2FM to his program which converts the stream to IMD format.  The code has been posted on github.  The imd file output of this utility can be viewed with another freeware utility, ImageDisk by Dave Dunfield. The display in ImageDisk showed 30 sectors of 256 bytes with 77 tracks, confirming our previous assumptions.

Finding the right disk

Many of the floppy disks contained in the set seem to have mostly data of clinical trials on them, judging from the file names and brief descriptions on the disk labels. We still did not know  how these data are actually organized and if there is much text contained within. We went back to the disks to see if we could glean any more meaningful information. Fortunately, there was one disk labelled “publications” which looked a promising candidate to test our colleague’s decoder.

Counterchecking

Ultimately it is really great if multiple paths produce reasonable results and it is possible to compare solutions with each other. Our colleague in Stuttgart came to the same conclusions as us regarding the particular used to format of the floppy disks:  He wrote “I had to kludge my program with the Kryoflux stream decoder. As far as I can tell, the floppy is in standard M²FM HP format with 30 sectors/track and 256 bytes/sector, ranging from sector 0 to 29.”

Our local colleague wrote some python scripts to to check CRC values and to extract text. He ran these on the disk we suspected of containing publications and to our delight we could read some of the content, as shown in Fig 1.

Around the same time, I received a decoded disk dump from Stuttgart. Using HPdir (a utility from the HP9845 Project), I can view the directory listing which matches exactly the original faded thermal print of the disk contents (Fig 2).

Conclusions

This project took a lot of patience and a lot of following hints and hunches to track down relevant information.  We also had a lot of luck and were aided enormously by the generosity of colleagues and strangers.

Trying to confirm the disk format and model of the original system took quite a while, until we finally made contact with someone who was actually involved with the scrapping of the system.

Various different levels of forensic science came into play; physical, logical, and higher level information were all needed to identify the system, the drive, the disk format so that we could finally extract some text.

We have not yet extracted other data content. This is much more complicated, as we now know that mostly scientific analysis work was carried out on the system with a lot of the data used in databases, spreadsheets and custom written analysis software.  Without the context or the software we cannot tell if we have the right numeric values, whereas it is easier for ASCII text.  As seen in the first figure, there are undecipherable characters within the text, these could be word processor markup or formatting, or something else entirely.

It would be possible to try to find publication(s) that match the text, so we can see to check if there are numeric values present. Knowing these values would help us then decode the original files. But that is work for another day.

Becoming more confident in the low-level encoding, it is time to look at the reproduction of actual bit streams. Our current knowledge, together with the directory listing, strongly suggests an HP system of the 1980s.  The call for input brought up some more information on possible disk formats. A colleague (in another city) who worked with HP machines and did some decoding of old floppies gave more hints (he actually wrote a decoder himself a while ago):

Here’s a rough sketchup of what my decoder does:
– Find the sync pattern (several 0x55)
– Swap the bits in each byte (i.e. MSB<–>LSB)
– Look for an address mark (clock pattern always 0x0E)

– 0x70: ID address mark
Calculate CRC16 (initial value 0xFFFF) over four bytes
– 0x50: data address mark
– 0xF0: defective track address mark
– 0xD0: ECC data address mark

– The ID address mark contains the following information:

0: cylinder
1: head in bit 7, sector in bits 5..0
2: CRC byte 1
3: CRC byte 2

Another colleague working on decoding some sample streams replied: “I’m not yet able to extract the sector data, but I can confirm that on several tracks that I’ve examined, the majority of intervals between flux transitions are at 2.0, 3.0, 4.0, and 5.0 microseconds, which matches the expectation for 8-inch M2FM.”

At first we started with the header information (short “sectors” after a sync pattern). We should be able to find the head and track information we have from the Kryoflux output. Sector information is not that easily available, because a track recording does not necessarily starts with sector 0 and usually contains more than just one rotation of the platter.

Identifying the Header Information

This information is contained in the short block after a sequence of 32 binary ones. We did a visual count in the representation of the logic analyzer software. Following directly after the pattern is the identification mark where we found the 0x70 address mark (after bit swapping and reading from right to left) and then the addressing information.

Track (or cylinder) number 31 could be read from the next byte. Both bytes produced expected results which validates the approach so far. For the sector some information was gathered too.

At the end of the data blocks (short and long)  some short patterns are visible (after episodes of binary zeros, mentioned by the other colleague) we presumed that could be a CRC.  A working CRC algorithm that would give us the chance to verify the data in that particular block.

Fig 1: The synchronization sequence is clearly visible, followed by ID and more information.

Track (or cylinder) number 31 could be read from the next byte. Both bytes produced expected results which validates the approach so far. For the sector some information was gathered too.

At the end of the data blocks (short and long) some short patterns are visible (after episodes of binary zeros, mentioned by the other colleague) we presumed that could be a CRC. If we had a working CRC algorithm we would be able to verify the data in that particular block.

Working on the Data

Now we finally come to the significant part. The start of the data sector is the same: initially 32 binary ones, then the marker. Directly after the marker the data stream starts. With the preceding information we have established the proper byte boundaries which should help for further investigation. This cannot be done manually.

Fig 2: The first filled in bits are the mark (0x50) followed by the data.

The data in this sector is a repeated 0xE3, 0x00 pattern. As the file listing does not include any tracks (cylinders) above 29 (starting from 0), we did not expect to find meaningful information here.

More on the disk format

During the different experiments more facts on the actual floppy disk format were gathered. The disks contain unusual (compared to the typical structure of IBM and PC style disks) 30 sectors instead of the expected 26. These sectors are interleaved and do not follow consecutively onto each other. The flux reading created by the Kryoflux tools contains in their track readings roughly five rotations of the platter. This offers the opportunity to compare different readings of the same sector to each other. This might help if in one reading is the jitter worse or the CRC did not produce confirmation.

Now we are looking forward to our colleague finding some time to extend his scripting for the clocking and coming up with a stream interpreter. An additional benefit would be checking of the CRC values to see if the conversion produced proper results. Now the rather simple structure of the disk (files written flat to sectors) comes in handy. No complex file system structure has to be taken into account to actually reassemble the files.