Berlin's Digital Archive Overhaul Hits a Snag — Thousands of Duplicate Images Clogging City Records

Berlin's Senate Department for Urban Development confirmed this week that a routine audit of the city's centralized digital asset system uncovered more than 40,000 duplicate image files embedded across municipal databases — a backlog that administrators say is complicating ongoing digitization work tied to the Berliner Digitalisierungsstrategie 2025-2030 framework. The discovery has forced a partial pause in public-facing updates to at least two major platforms used by residents and planners alike.

The timing matters. Berlin is mid-way through an aggressive push to consolidate its fragmented administrative IT infrastructure — a project that affects everything from BVG route documentation to the Stadtentwicklungsplan housing registers used by planning offices in Mitte and Neukölln. Duplicate image files, while mundane-sounding, create cascading problems: bloated storage, broken metadata links, and search results that return the same photograph or building scan dozens of times, making records harder to use and audit.

Where the Problem Is Showing Up

The issue is most visible at two institutions. The Landesarchiv Berlin, headquartered on Eichborndamm in Reinickendorf, has been working since January 2026 to migrate roughly 2.3 million digitized records into a new unified content management system. Staff there discovered that automated scanning of pre-war housing documents and street-level photography had generated duplicate image entries at a rate administrators had not anticipated — in some document batches, duplication ran as high as one in five files. The Stadtmuseum Berlin, which manages collections across multiple sites including the Märkisches Museum near Köllnischer Park, reported a similar problem in its publicly accessible image portal, where duplicate entries for the same artefact photographs began surfacing in searches after a January software migration.

For residents trying to use the Stadtentwicklungsplan online portal to check planning applications in their Kiez — a process that became more urgent as the SPD-led Senate coalition debated rent cap extensions earlier this year — duplicate image entries on property records have slowed load times and, in some cases, returned conflicting file dates for the same building photograph. That is not a trivial inconvenience in a city where housing disputes frequently hinge on the precise documentation of a building's condition and permitted use.

What's Being Done — and How Long It Will Take

The Senate Department has contracted with Berlin-based software firm dataspot GmbH, which specialises in public-sector data governance, to run a deduplication process across the affected systems. Work began the week of June 30 and is expected to continue through at least late August. The project involves hashing algorithms that compare image files at the pixel level, flagging near-duplicates as well as exact copies — a distinction that matters when scanned documents have been rescanned at different resolutions over the years.

Cost estimates for the full cleanup have not been made public. However, the broader Berliner Digitalisierungsstrategie carries a budgeted envelope of €310 million through 2030, of which data quality management is one identified expenditure category. City IT officers have previously estimated that unresolved data redundancy across Berlin's 41 administrative departments costs the city measurable processing time each year — though precise per-department figures have not been published in the current budget cycle.

For residents and researchers who rely on these databases — historians at the Humboldt-Universität who access the Landesarchiv remotely, or architects checking Mitte building permits on FIS-Broker, the city's geographic information portal — the practical advice for now is straightforward: if a search returns what appears to be duplicate results, note the file reference number rather than the image thumbnail, since reference numbers were not affected by the duplication error. The Senate Department says a public status page tracking the deduplication progress will go live on berlin.de by July 11.