Case Belspo

Challenges

The challenges our solution meets at Belspo:

• Input of large volumes of digital files, both in terms of number and file size (around 50 terabytes daily)
• Providing a common solution to the various FWIs that each have their own needs and requirements in terms of metadata structure and input
• Secure solution with redundant storage of all files and metadata in two separate data centres.
• Very long term storage on tape carriers with an initial capacity of 2000 terabytes.

Solution

Zeticon went through the following steps in consultation with VIAA. The analysis of the solution which was needed because it involved millions of documents. In addition, the implementation took place whereby Zeticon had its input with the installation of the hardware. We worked together with SMALLS to get the project started.

During the implementation, in addition to the infrastructure, time was taken to set up a very extensive range of workflows. Examples such as the unzipping of files into one file or the intake of hundreds of thousands of small documents were examined.

Consultations are held at very regular intervals (minimum monthly) to take further steps. There are many other possibilities arising from this central vision that are considered in a continuous process.

Belspo typically imports a large number of documents, for which MediaHaven's search and database are crucial. A federated search listed by FWI is one of the recent focal points. The content is a combination of documents and audio-visual material, to which this content is added in various ways (according to institution). This can be directly in the application, but also via FTP or the API.

The hardware architecture of the LTP platform consists of:

• The primary data centre (main) and the redundant data centre (backup);
• A temporary buffer memory (hard disk cluster) in the two data centres;
• Storage for sample files (hard disk cluster) in the two data centres;
• very long term storage on tape carriers in the two data centres;

The hardware for this project was set up together with SMALS, with Zeticon taking care of the software management. Needless to say, the cost of the project is high. The project has been delivered since 2016/2017 and remains in a constant operational phase with adjustments made where necessary.

Standard support for the following interfaces and protocols is very important, as it is for VIAA as well as other customers:

• SAML 2.0 for authentication
• JSON
• Very extensive RESTfull API
• OAI-PMH support
• Import with or without sidecar (metadata via XML)
• Support METS, Dublin core for metadata profiles
• QA and live environments
• Various input options
• PREMIS logging