Michela Vignoli

SCAPE is proud to look back at another successful project year. During the third year the team produced many new tools, e.g. ToMaR, a tool which wraps command line tools into Hadoop MapReduce jobs. Other tools like xcorrSound and C3PO have been developed further.

This year’s All-Staff Meeting took place mid-February in Póvoa de Varzim, Portugal. The team organised a number of general sessions, during which the project partners presented demos of and elevator pitches for the tools and services they developed in SCAPE. It was very interesting for all meeting participants to see the results achieved so far. The demos and pitches were also useful for re-focusing on the big picture of SCAPE. During the main meeting sessions the participants mainly focused on take up and productization of SCAPE tools.

Another central topic of the meeting was integration. Until the end of the project the partners will put an emphasis on integrating the results further. To prove scalability of the tools, the team set up a number of operative Hadoop clusters instances (both central and local), which are currently being used for the evaluation of the tools and workflows.

Another focus lies on the sustainability of SCAPE tools. The SCAPE team is working towards documenting the tools for both developers and users. SCAPE outcomes will be curated by the Open Planets Foundation until the end of the project and will keep them available.

In September 2014 SCAPE is organising a final event in collaboration with APARSEN. The workshop is planned to take place at the Digital Libraries 2014 conference in London, where SCAPE will have its final, overall presentation. The workshop is directed towards developers, content holders, and data managers. The SCAPE team will present tools and services developed since 2011. A special focus will lie on newly and further developed open source tools for scalable preservation actions; SCAPE’s scalable Platform architecture; and its policy-based Planning and Watch solutions.

The second project year of the integrated research project SCAPE has just finished. It is time to look back at the most interesting developments and significant results of this period.

Last December, SCAPE successfully organised its first public training event: Keeping Control – Scalable Preservation Environments for Identification and Characterisation (supported by the European Capital of Culture in Guimarães, Portugal). The event was well attended, and the participants gave useful and very positive feedback to the trainers and organisers. The training material from the event is available on the OPF wiki (http://wiki.opf-labs.org/display/SP/Resources+-+SCAPE+Training+event+-+Guimaraes).

Communication and cooperation between the technical teams was increased during this project year, in order to integrate developments and results in SCAPE. The 1^st Platform Release was an important milestone: the SCAPE components included in this release are listed on the OPF-wiki (http://wiki.opf-labs.org/display/SP/First+Platform+Release). The current version is a first step towards the scalable infrastructure which the project is developing. In 2013 the Platform team will test and enhance the current version of the platform. A short paper and a presentation explaining the architectural overview of the preservation platform are available on our project website (http://www.scape-project.eu/publication/an-architectural-overview-of-the-scape-preservation-platform).

A number of SCAPE components have been developed and adapted in order to address scalability, automatisation, and quality assurance of stored data.  Two examples of SCAPE quality assurance tools that saw their first public releases in 2012 are Pagelyzer (comparison of web pages) and xcorrSound (comparison of sound files).

All SCAPE tools have been evaluated against the SCAPE scenarios and their current status assessed. A report on this gap analysis is available on the project website (http://www.scape-project.eu/news/d10-2-gap-analysis-on-action-services-tools-and-scape-platform-and-testbeds-requirements-draft). A first evaluation of SCAPE workflows has taken place as well. The workflows can be accessed on myExperiment (http://www.myexperiment.org/workflows?query=scape), and the evaluation report is available on the project website (http://www.scape-project.eu/news/d18-1-first-evaluation-report-draft).

Significant progress has been made towards extending the functionality of the planning component Plato, and adapting it to meet the objectives of SCAPE. A number of new features have been added to Plato in order to integrate information from myExperiment (e.g. about migration services), automated content profiling, and policy-awareness.  A public instance of the newest version of Plato can be accessed at www.ifs.tuwien.ac.at/dp/plato.  A first version of our machine readable Policy Model can be accessed at GitHub (https://github.com/openplanets/policies). Finally, a first version of the automated watch component SCOUT has been released as well (https://github.com/openplanets/scout).

During the coming project year, the different SCAPE teams will work towards releasing the final version of the SCAPE preservation platform, and improving the SCAPE components in order to meet requirements for scalability. Dissemination, training and sustainability activities will continue as well. Planning for Year 3 has started at a very successful All-Staff Meeting, held in February 2013 in Paris. The agenda included a large number of sub-project and work package meetings, as well as of general cross-project sessions. During a workshop the project members identified the most successful and productive project results so far. The workshop results were very useful in gaining an overview of the current status of developments and to re-focus current work on the main project goals. The SCAPE team is looking forward to another very productive project year!

(pictures by Sava Cajetinac)

SCAPE – what is it about?

SCAPE is an integrated research project co-funded by the European Union under the FP7 ICT program. It is running since February 2011 and has a total duration of 42 months. The SCAPE Consortium brings together a broad spectrum of expertise from memory institutions, data centres, research labs, universities, and industrial firms. Sixteen European institutions are cooperating to develop
solutions for the long-term digital preservation of large-scale and heterogeneous collections of digital-objects. Their aim is to develop scalable services for efficient and automated preservation planning and the execution of preservation actions of large (multi-Terabyte) and complex data sets.

SCAPE’s contribution to digital preservation

The volume of digital content worldwide is increasing exponentially. This fact demands that preservation activities become more
scalable and automated. However, current solutions for the preservation of digital object collections fail when applied to very large or complex sets of data.

SCAPE is devoted to enhancing the state-of-the-art of long-term digital preservation by developing an infrastructure and tools for scalable preservation actions (the SCAPE Platform and Components), and a framework for automated, quality assured preservation workflows. Additionally, these components will be integrated with a policy-based Preservation Planning and Watch System.

Benefits for Content Holders

The scalable preservation environments that are being developed by the SCAPE project team will enable organisations to keep pace with the rapid growth of digital collections by providing a new highly scalable architecture and reducing the need for human intervention. It will help ensure that the organisations’ preservation actions have been effective, and make sure that increasing regulatory obligations of the organisations are being fulfilled. Finally, it will reduce the risks to the organisations’ digital material.

The Project: Short overview

In order to meet the overall project goals, the SCAPE Consortium has conceived a project structure that enables a high level integration of the results from the several research and development work packages.

The Testbeds sub-project is the primary driver for the rest of the project in that it determines the use case scenarios, defines the preservation workflows, and evaluates the platform and the tools created in other parts of the project. The SCAPE Testbeds define specific preservation issues (described by the scenarios) with a special focus on three application areas: Large-scale digital repositories, Research Datasets, and Web Content.

The Preservation Platform sub-project is developing platform reference architecture for scalable preservation environments. The SCAPE Preservation Platform is an extensible infrastructure that provides reliable storage of voluminous data objects and records, and which supports the parallel execution of preservation tools and workflows close to the data. Additionally, it provides a scalable backend which can be attached to different data management systems.

The Preservation Components sub-project delivers the preservation components that will be executed on the platform. These tools will enhance the functionality of the digital preservation system in scalability, functional coverage, and quality.

The Planning and Watch sub-project develops automated components that address the bottleneck of decision processes and processing information required for decision making. The Watch Component (SCOUT) gathers information relevant to digital preservation activities (e.g. from repositories, technical registries), and provides an automated monitoring system. The Planning Component (that is building upon PLATO supplemented by a machine interpretable policy model) supports decision makers in producing automated preservation plans based on individual digital collections, preservation policies, previous preservation plans, and controlled experiments. The Planning Component integrates with SCOUT and the SCAPE repositories.

Coordination and Takeup Sub-projects

To make sure that the project delivers high-quality results, the research and development work packages are being coordinated by the Cross-Project Activities sub-project. Additionally, the Takeup sub-project ensures the regular dissemination of these results, and guarantees sustainability and impact of the outcomes beyond the project.

SCAPE: Scalable Preservation Environments

In the future, SCAPE will transform the manner in which content holders safeguard their digital content. The project results will increase confidence in the long-term accessibility and integrity of the content holders’ collections (at the European, national, and organisational levels). Furthermore the project SCAPE will have a profound impact on the way that digital repositories are designed and built in order to enable scalable preservation services.

For more information about the SCAPE project and its publications please consult

www.scape-project.eu/about
www.scape-project.eu/category/publication
www.openpreservation.org/projects/scape.

SCAPE newsletters:
www.scape-project.eu/category/newsletters

Want to get information at first hand?

Subscribe to the SCAPE newsletter!

After the first project year SCAPE members can look back at an outstanding project start and an intense phase of integrated project work. First results have already been achieved and spread to the community at several conferences and through different channels on the web.

In the first project year six deliverables (five of which were public) have been delivered on time to the EC. They have all been accepted by the EC in succession to the first year review. Three additional reports have been published on the SCAPE website. Additionally, 15 scientific publications presenting results developed within the SCAPE project have been published in journals and conference proceedings.

Some highlights of the first project year:

• An experimental cluster for development work on the preservation platform has been deployed.
• Several applications and components have been released (e.g. a Prototype for command-line execution of Hadoop applications, the SCAPE Action Catalogue, the Akubra HDFS adaptor).
• 22 SCAPE Scenarios have been developed. Here Datasets, Issues, and Solutions from SCAPE content providers are documented.
• Numerous experimental Taverna Workflows have been developed and tested.
• 52 action services are online and available for these Taverna test workflows.

SCAPE results can regularly be followed through the project website, OPF blog posts and the SCAPE Twitter feed.

We are pleased to announce that SCAPE has passed its first year review with flying colours. Our Project Officer stated that the project has achieved excellent results, that all objectives for the first year have been achieved, and that there were some particularly strong points. The initial feedback from the review panel was also quite positive. The panel noted the quality and number of first-year demonstrations that were also clearly the result of collaborative work. SCAPE’s Technical Coordinator was praised for an excellent overview of the technical aspects. The extent of re-using past project results in SCAPE was also rated positively. Finally, the SCAPE project management was judged to be highly professional.

The SCAPE team is very pleased by the success that it has achieved in the first project year, and is keen to fulfil the high expectations of the commission and the community for the second year.