Berlin's Digital Archives Push Confronts a Stubborn Problem: Thousands of Duplicate Images Clogging City Records

Berlin's centralised digitisation programme hit a concrete obstacle this week when administrators at the Landesarchiv Berlin confirmed that an internal audit, completed on July 2, had identified more than 40,000 duplicate image files spread across shared government servers — a problem that has been quietly degrading the efficiency of the city's digital records infrastructure for at least three years.

The issue matters now because the SPD-led Senate has staked significant political capital on getting Berlin's bureaucratic backlog into a unified digital system by the end of 2027. Duplicate image files — identical or near-identical scans filed under different reference numbers — eat storage capacity, inflate indexing costs, and cause retrieval errors that slow down planning approvals, housing applications, and infrastructure tenders. With housing waiting lists in Mitte and Neukölln stretching past 10,000 registered applicants each, any friction in the document pipeline has real consequences for real people.

Where the Problem Showed Up

The audit covered holdings at the Landesarchiv on Eichborndamm in Reinickendorf and the digital document management systems used by Senatsverwaltung für Stadtentwicklung, Bauen und Wohnen on Württembergische Straße in Wilmersdorf. Both repositories had been ingesting scanned documents from multiple city departments since 2021 without a shared deduplication protocol. The result was predictable: planning maps, building permits, and aerial survey photographs were saved repeatedly as different teams submitted overlapping batches.

The BVG, Berlin's public transport operator, ran into a related headache during its own infrastructure documentation push. Internal project files for the U-Bahn line expansion planning — work that feeds into the broader BVG investment programme backed by federal Deutschlandticket revenue — contained hundreds of duplicate construction-site images that had to be manually sorted before tender documents could be finalised. A BVG communications statement released this week described the deduplication work as a necessary but resource-intensive step in the preparation of its multi-year capital expenditure records, without giving a precise cost figure.

What the City Is Doing About It

The Senatsverwaltung für Inneres und Sport, which oversees Berlin's IT infrastructure strategy, has commissioned a tender for automated deduplication software. The procurement notice, published on the Berlin Senate's official vergabemarktplatz portal on July 1, sets a contract value ceiling of €480,000 and a delivery deadline of March 31, 2027. Bidders have until August 15 to submit proposals. The software is expected to use perceptual hashing — a technique that detects visually identical or near-identical images even when file names differ — to clear the backlog and establish ongoing prevention protocols.

The scale of the problem is not unique to Berlin. Hamburg's Staatsarchiv reported in 2024 that a comparable digitisation audit there found redundant file rates of roughly 12 percent across scanned holdings — a figure that gives Berlin's administrators a rough benchmark as they work through their own numbers. Berlin's 40,000-file count has not yet been translated into a percentage of total holdings, and the Landesarchiv said a full assessment would take several more weeks.

For Berlin's startup and tech sector, which has increasingly pursued city contracts in the govtech space — several firms cluster around the Factory Berlin campus on Rheinsberger Straße in Mitte — the procurement notice represents a genuine commercial opening. Companies offering AI-assisted document management have been lobbying Senate departments since at least 2024, and the deduplication tender is the first to reach formal procurement stage.

The practical upshot for residents and businesses dealing with Berlin's planning and permits system is that relief, if the software contract goes smoothly, is still the better part of a year away. Until March 2027, city staff will continue handling duplication issues manually on a case-by-case basis. Anyone submitting building applications or housing-benefit paperwork through the Dienstleistungszentrum portals is advised to keep certified hard copies of every submission, since retrieval errors remain a documented risk in the interim period.