Berlin's Digital Archive Crisis: The Key Decisions Ahead on Duplicate Image Replacement

Berlin's public institutions are sitting on a problem that has been quietly compounding for years: tens of thousands of duplicate digital images are consuming server space, distorting search results, and slowing down databases that residents depend on daily. Now, with the SPD-led Senate pushing a broader digital modernisation drive through 2027, administrators across the city must decide how to replace legacy duplicate files — and who pays for it.

The issue is not abstract. At the Stadtentwicklungsamt, the city's urban development authority, housing applicants navigating the online WBS portal have encountered broken listings and repeated images for the same properties for months. Meanwhile, at the Landesarchiv Berlin on Eichborndamm in Reinickendorf, archivists have flagged that digitisation projects running since 2019 have created significant file duplication across the photo collections cataloguing the city's postwar reconstruction.

Why the Decision Can't Wait Much Longer

Two forces are pushing this toward a crisis point in the second half of 2026. First, the Berlin Senate's IT infrastructure budget cycle closes in September, meaning funding applications for deduplication software and storage migration must land before the summer recess ends. Second, new EU data governance rules taking effect across member states in early 2027 require public bodies to maintain audit-ready digital asset registers — something impossible to do cleanly when thousands of images exist in multiple versions across disparate servers.

The scale of the problem varies by institution, but it is not trivial. Across comparable mid-sized European municipal archives — including those in Vienna and Hamburg — deduplication exercises have typically identified between 18 and 35 percent of stored image files as redundant copies, according to published findings from the European Commission's digital public services unit. For Berlin, which accelerated its digitisation push during the Covid-19 period between 2020 and 2022, the exposure could be significant.

At Tempelhof Projekt GmbH, the public developer overseeing the Tempelhofer Feld site, staff are already using a third-party asset management platform to flag duplicated architectural renders and planning photographs from the ongoing development consultation process. That system cost roughly €40,000 to license for a two-year term, according to publicly available tender documents posted to the Berlin procurement portal. That price point is seen internally as a benchmark for what Senate departments might expect to pay for comparable tooling.

Three Decisions That Will Shape the Outcome

The choices ahead break down into three concrete questions. The first is whether Berlin's Senate Department for Digital Transformation, based on Berliner Str. in Wilmersdorf, will commission a centralised deduplication system covering all Senate departments or allow each authority to procure its own solution. A fragmented approach risks reproducing exactly the siloed structures that created the duplication problem in the first place.

The second question involves the Landesarchiv specifically: heritage images require human curatorial sign-off before any file is deleted, because automated deduplication algorithms occasionally flag near-identical but historically distinct photographs as redundant. Getting that sign-off at scale requires either additional archival staff or a supervised AI workflow — neither of which exists yet in the current Landesarchiv budget for financial year 2026.

Third, and most immediately practical for residents, is what happens to the WBS housing portal. The Investitionsbank Berlin, which administers parts of the affordable housing subsidy system, is scheduled to roll out a refreshed applicant interface in the fourth quarter of 2026. Whether that rollout includes a proper image asset audit — removing duplicated property photographs that currently mislead applicants about available units in districts like Marzahn-Hellersdorf and Spandau — is still unresolved.

Advocates for open government data, including those working through the Berlin chapter of the Open Knowledge Foundation, have argued publicly that the Senate should publish a clear timeline for the deduplication decisions by September 15 at the latest. Without that anchor date, the risk is that individual departments make incompatible choices, leaving Berliners with a patchwork of half-cleaned databases rather than the coherent digital infrastructure the city has been promising for three years.