Translation of Photographic Archive into Algorithmic Time

Wolfgang Ernst

Still time? Analogue daysand photographic in/formation

Let us look at a digitised  photography in a media-archaeological way. Phenomenologically its image quality (provided that the sampling rate has been sufficient) looks more or less like the analogue original, but chrono-technically it embodies a totally different essentiality. Even if it is subject to physical and chemical entropy, the analogue photographic print remarkably endures, while its digital version is a function of data arrays which need to be electrically powered or even refreshed permanently. While the analogue print is embedded in physical time, the digital image reveals its processual character and is thus generated by its own internal temporal mechanism (oeprative algorithms); it has to be re-generated out of computer storage. The photographic punctum corresponds with the indexical temporal momentum, but there is no "still photography" in digital representation.

There are two approaches to the conservation of analogue photography. The one cares for preserving the physical, especially chemical and electro-magnetic properties of the physical and chemical storage medium (all media are material in the first place). The other, somewhat opposing approach is to preserve medium-based memory as information, up to the extreme point of view that the material body might be abolished after its essential transformation into its pure binary information units: "We no longer collect the carriers, clay tablets, books or floppies, just the information". [1] But to which degree does the archival authority of an archival record still depend in its material physical embodiment?

Imagine two photographs of one and the same ancient chapel of St. George in the mountains of Bulgaria. One photograph has been developed upon chemical basis, the other as a digital snapshot. Analog photography itself is indifferent towards the tempor(e)ality of its present or "historic" referent; its historicity lies rather in the entropy of its own physical state. Against this, digital photography is a-temporal, carrying the temporal trace not in its information (which is its binary essence), but in the hard- und software into which this information is embedded.

Evidently a different nature opens itself to the digital camera than opened to the naked lense in the traditional photographic apparatus. [2] Walter Benjamin's term "technische Reproduzierbarkeit" turns into "digital reproduction", a mathematisation    of the photographic process, a different archive.

As has been accentuated by the media philosopher Vilém Flusser: A code has been obtained that comprehends images. [3] This leads to the option of creating new images out of the code language which can be activated online. Media archaeology deals with such techno-mathematical logic, not just with origins and previous image engineering in the traditional sense. An archive of different temporalities opens.

For the oldest signal-based medium in the technical sense, photography, in 1859 Oliver Wendell Holmes pointed to the fact that the symbolic trade of media and material was introduced by photography:

"From now on, form is separated from material. In fact, the material in visible objects is no longer of great use, except when being used as a model from which the form is constituted. Give us a couple of negatives of an object worth seeing ... that's all we need. Then tear the object down or set it on fire if you will (...) the result of this development will be such a massive collection of forms that it will have to be arranged into categories and placed in great libraries". [4]

What was form in analogue days, becomes information in digital space. Once the light signals which have been chemically engraved on material carrier has been transformed into digital, immaterial information, it can be (virtually lossless) "migrated" from one storage computing system to another.

 Photo-archival preservation: from materiality to information

We observe the transformation of the material storage into an archive in electronic motion, in electromagnetic ephemerality and latency. The gain of flexibily and computability, though, is paid for with a dramatic loss of durability. Permanence and archival endurance thus is not achieved in the traditional way any more (which has been monumental fixation, stasis so far), but by dynamic refreshing.

When a few years ago the architectural building of the Cologne Municipal Archive collapsed due to Underground construction works, it became apparent that most records, though having become dirty and mutilated, materially survived this catastrophe, astonishingly resistant agains the pressure of stones and water. In a similar way the first-generation audiovisual storage media turned out to be surprisingly resistant against temporal entropy (like Edison-cylinders and gramophone records, as well as daguerreotypes, photographic negatives and film on celluloid). More delicate is the destiny of cultural memory based on electromagnetic storage; digital media, finally, tend to divest themselves completely from their material embedding - loosing the physical ground by becoming technically "virtual".

Traditional storage media have been physical inscribed (graphein in its old Greek sense): By writing the information to be stored literally in-forms the device.[5] Latent storage devices such as magnetic tape for audio and video, on the contrary, only reveal their memory content in the dynamics of the electro-magnetic field as induced signals - an "archive" which human eyes cannot decipher any more immediately. Analogue electronic storage media indexically take place in a sphere which is different from the scriptural regime of the classical archive, but the symbolical regime, on the level of alpha-numeric codes, unexpectedly returns in techno-mathematical machines. This re-turn is a temporal figure which cannot be reduced to the linearity of cultural history; we are confronted rather with a kind of recursion. With computed binary data, the "archival" symbolical regime returns into audio-visual media themselves, but in a different way which is numerical. [6]

In trans-photographical archives as data spaces the message of the medium is the alpha-numerical code. This implies a profound mathematisation (instead of inconisation) of what used to be called an "image". A digital photography is no material light inscription any more, but its numerical information - as becomes evident when the "core dump" mode is chosen for its representation on display.

 Digital regimes: The archival from within photography

 Let us refer to the epistemological notion of "archive" as expressed by Michel Foucault's Archéologie de Savoir (1969): Which rules defines what kind of photographic memory can be expressed and remembered (that is: stored) at all? It is not only human archivists any more, but in a higher degree than ever it is technologies upon which the readability of such documents depends. The archival record has become techno-mathematical sublime in electromagnetic latency - being there, but not accessible to human senses any more. All of the sudden, the Foucauldean archive turns out in digital photography. With the digitization (thus mathematisation) of photographic matter into information its temporal essence transforms as well: "Time no longer has physical meaning." [7]

Analogue photography by its very materiality inscribes traces of time, whereas in digital photography, the temporal index becomes a para-text, in fact: meta-date without physical evidence of aging.

In current media art, the "archaeological" use of anachronistic media like 16mm film appears like a retro-effect against digital atemporality - an archaic counter-practice, archival resistance, a nostalgia for the passing of time. [8]

The digital photograph preserves the iconic quality while loosing the indexical trace. Or rather, indexicality itself here is of a different kind. While the analogue photographic print keeps a physical trace of the past, recording the light intensities, the digital pixels keep a schmatic, mathematically abtracted relation to their generating (and then sampled) analogue signals - a diagrammatic indexicality.

In a Deleuzian terminolgy and re-reading of Foucault, the archive presents a kind of diagram. The multi-media "archive", rather, represents an operative diagram, a diagramatic machine, still topological (graphs, nodes) but with the additional dimension of algorithmic temporality.

Photographic signal inscription for decades has been a kind of analogue measuring of time, as opposed to the familiar symbolic registration of past events in alphabetic writing. The non-linearity of photo-archival memory separated this aggregation from the smooth continuity effects of historiographical narrative.

With advanced digital media, both regimes - the symbolical order and the signal-based "real" - miraculously converge: The computer, by digital signal processing (DSP), is capable now of emulating all "analog" happenings in the real physical world (which before only the "analog media" like gramophone and video could perform) by means of algorithmically processing the strictest of all symbolic, in fact: alphabetic regimes, which is the binary code. This results in the ultimate algorithmic temporalisation  of archival photographic memory.

Digital compression is the basis for the online streaming of cinematographic arrays of photographic images - in fact a delicate micro-archival calculatios. Only parts and sections of an image are updated at a temporal moment. MPEG technologies divide each frame into small blocks of pixels in order to analyze changes from one frame to the next. A group of frames is established around one key frame at intervals. [9]

To achive real time transmission of photographic images in the Internet the intervention of dynamic algorithms for effective data compression is required "to reduce the spatial redundancy among the picture elements and to reduce the temporal redundancy between successive frames". [10] Such predictive coding is not a visual trace from the past any more but more time-critically a trace of the immediate future." A sky could be mostly blue. Rather than transmit an exact replica of the sky, why not use an algorithmic process that transforms the blue sky into a quasi-statistical summary of the spatial distribution of blueness?" [11] This allow to compress those components of an image that are most perceptually redundant to human perception. The basic photographic format, tis frame, is being deconstructed. Rather than the single image being the elementary unit of photography, the block becomes its basic component.

 New "anarchival" options in re-membering digital images

 A kind of "anarchival impulse" engenders photography collections in terms of mathematical stochastics once images exist have been translated into the digital regime. [12] In virtual memory space, new options of sorting images arise, different from categorical logocentrism and indexing by metadata, in fact: arrangements which arise from within the digital image itself (so-called "imaged-based image retrieval“). [13]

Tagging and meta-dating of images is a supplementary, belated symbolical operation applied to images. Automated sorting of images to a large degree still depends on such annotation: "Computers can help us. But only after we help them first by feeding images descriptions." [14] Since once an image has been turned from a physical carrier into information by the act of digital scanning, is transforms into a mathematical representation devoid of semantics. The computer has to be trained in order to gain icono-logical knowledge; to teach the computer human "thinking" has been the dead end of Artificial Intelligence. But let us turn this argument upside down. The apparent computational lack, the "semantic gap" which separates the Turing machine from human understanding, can be interpreted as its virtue, since it opens an aesthetics of parametrical sorting and archiving - opening unforeseen spaces of visuality.

Walter Benjamin once identified the ephemeral human awareness of similarity as the essence of lived experience - a mixture between vague impression and intentional perception. [15] With, for example, the progressive sorting of distributed pixels according to colour similarity, an anarchival or rather para-archival impulse can be identified in the algorithms of similarity-based image retrieval.

Let us, in that sense, imagine "experimental archives" different from the well-organised institutional archive. Quantised (digitised ) images can be transformed into a vast image bank which, once unified as data-set, can be subjected to image-based search operations such as matching of similarities, object feature detection, statistical colour value comparison etc. New kinds of search engines not only answer the needs of knowledge retrieval but develop into a creative art of the archive.

Media arts have already become avant-guarde in that sense by experimenting with new forms of access to image down to its single pixels. [16] The strict basis for such experiments, though, is algorithmic knowledge.

Image-based search for images takes information itself as criterium in the order of images. The loss of material authenticity in technomathematical reproduction in return leads to arriving at another level of abstraction; its mathematical intelligence is based on technically standardised, unified alphabets. In fact there is nothing really anarchic in the digital world, since the alphanumeric regime - which is the symbolic regime - is always about order.

Between image-based sorting of photography and logocentrism (George Legrady)

In his Kunstgeschichtliche Grundbegriffe (1915) the art historian Heinrich Wölfflin once aimed at formal criteria for sorting  art historical images according to criteria like "open" vs. "closed" form. [17] Today, this vision can be realised  by automatic image-based image grouping. [18] Such a clustering sucessively liberates image configurations from word-based tagging. Even commercial digital images sorting software for private photographies sometimes offers the display of histograms (diagrams displaying the statistical distribution of colour in images); this is a perfect training in image-immanent navigation of the visual archive.

In optical scanning, the computer does not recognize a photographic "image" in its cultural (thus human) sense, but rather its elementary parameters: statistical colour distribution, edges, lines, shapes et cetera. Stochastic rather then library-oriented, classification-based sorting of images thus becomes feasable. At the same time, digitisation of images results in an ultimate addressability of each single picture element, the so-called pixel. Adressability is a central characteristic of the archival operation; thus we can say that by digitisation the image becomes essentially archival. [19]

Even cinematographic movies which - as we know - consist of sequences of discrete photographic frames - can thus be transformed into a vast searchable data-set.

The correctness of computer memory is its essential lack when compared to human remembrance operations which rather distort memories; according to the inventor of the graphical user interface in computing, Licklider (1960), the human is a "fuzzy, noisy device", but in turn gifted with the capability of parallel signal processing. From that results a different attitude towards image collections:

"Fuzzy" computer-sorting will begin to make useful comparisons of similar (but not identical) images on the basis of new protocols. Or should we rather "work harder on the alphanumeric labelling and keywording of pictures (...) aided by re-born analogue machines"? [20] Shall we aim at closing the "semantic gap" between the anarchivic element within humans and computing? Shall we train computers to behave counter-logically?

Close to such neuro-aesthetic insights, the traditionally “dialogic“ rhetoric of the archive is currently being replaced by operational archival interaction, as illustrated by Pockets Full of Memories (, an online and museum installation by the media artist George Legrady in which the audience creates an archive by contributing a digitally scanned image of an object in their possession during the exhibition visit. [21]

In multi-media space, the act of re-activating the archive can be dynamically coupled with feedback. Interaction is an aspect Bertolt Brecht pointed at already in the 1920s for the emerging medium radio, insisting that it can technically - when provided with a feed-back channel - be used in a bi-directional way by the receivers to communicate instead of being unilaterally subjected to central broadcasting. [22] The unidirectional communication of books still dominated the user experience. With different hierarchives, a network is not a text any more, rather an archi(ve)texture.

The "dynamic archive", in fact: the sorting engine of Legrady's installation is based on a self-organizing map, known in computer science as the Kohonen algorithm. The Kohonen algorithm corresponds with neuro-scientific evidence: "The self-organizing map captures some of the fundamental processing principles of the brain, especially of the experimentally found ordered maps in the cortex." [23]

It is in terms of an auto-associative network that an electronic switch-principle for visual memory is being discussed for explaining image generation in the brain, especially within the cortex region. Computer science adopts this by modelling auto-associative networks; these have properties that are comparable with visual memories. Within a matrix of parallel-switched neurons their synoptic links react on themselves in loops, thereby starting to store content.

The self-organizing map in Legrady's installation translates the key-words (semantic information) and object description and turns them into numbers; this is how the mathematically determined organisation happens. Many of the given meta-data influence the positioning, for instance, the date, possibly the object's origins.

In terms of informational communication theory theself-organising map is a semantic memory model which is dynamic, associative and consists of adaptive prototypes. This correponds with Vannevar Bush's insight which became essential for the development of hypertextual knowledge: "Memory is transitory." [24] Bush in 1945 formulated his design of a Memory Extender (MEMEX), a memory machine which is not oriented at the artificial taxonomy of libraries but at the human brain functions which operates less logically but associative.

 Temporalising the archive: Fromspace-basedtotime-basedarchives

 The art historian Aby Warburg once created a dynamic collection of photographic reproductions of historic works of art called the Mnemosyne Atlas ( which aimed at tracing the rather unconscious tradition of visual expressions in occidental culture. Warburg thus established, between the two World Wars, a visual, photography-based archive of gestic expressions (so-called pathos formulas)in Western art history, in the form of his Mnemosyne-Atlas (a kind of visual encyclopedia where the reproductions, provided with numbers, could be constantly re-arranged and re-configurated). But although Warburg conceived of his chart sequentially, even there the apriori of this pictorial memory is still the order or the library. It is the famous Warburg file catalogue (Zettelkasten) which translates both texts and images in alphanumerical notations which then allow for the hypermedia-like linking of visual and verbal information.

Philippe Alain-Michaud describes Warburg's Mnemosyne panels as functioning like screens on which the phenomena produced in succession by the cinema are reproduced simultaneously. [25] Archiving of photographies "on the line", most literally, re-calls the cinematographic stripe and reel of celluloid indeed.

But the basic unit of Warburg's picture tables was still the photographic frame. With digital sampling of images, all of the sudden photograph can be literally addressed down to the single pixel.

Addressing and sorting visual images remains an urgent challenge not only because of the commercial potential of image archives. While digitisation does not necessarily guarantee better image quality, it does offer the option of addressing not only images frame by frame, but even each single picture element (pixel). Images and sounds thus become calculable and can be subjected to algorithms of pattern recognition – procedures which excavate unexpected optical and even "sonic" statements out of the photographic archive. In the media installation Voice of Sisyphus under George Legrady's artistic direction, methods of digital archaeology (operative image analysis) have been used to sonify the image-as-memory itself. A black & white photographic image from the 1970s displaying a hotel scene "At the Bar" is filtered by a computer program which then reads the segments and produces sounds out of these data resulting in a continuously evolving composition. [26] This is no deliberate metaphorisation, but a algorithm-based transformation of the archive (in Foucault's sense), giving a voice to the photographic image.

With effective algorithms, for the first time, the photographic image archive can organise itself not just according to meta‑data, but according to criteria proper to its own data structure: an endogenic visual memory in its own medium. By translating analogous photographic images (including film) into digital codes, not only do images become addressable in mathematical operations, their ordering as well can be literally calculated.

While the traditional photographic archive (such as André Malraux' musée imaginaire) still represents a spatial order ("l'espace de l'archive"), today the online image archives themselves take place in time. [27] Dynamic access to image archives needs a flexible tool which allows for the coexistence of different orders without destroying the existing database structure.

The archive itself gets in motion; the storage of large amounts of photographic objects results in new types of transmission, compression and retrieval which are based on differentiation like the send-on-Delta sampling which only registers decisive alterations to sequences of similar images. Dynamic access now replaces the static classification of the traditional logo-centristic catalogue, just like statistical probabilities have replaced particular knowledge in information theory, and pattern recognition replaces individual identification.

 "Social archives" in Web 2.0?

 With(in) the World Wide Web and with the emergence of such "social media" photographic archives like Flickr, traditional archiving practices which had been restricted to authorised archivists and users change towards appropriation and migration of photographic records and generate new forms of image archives - be it artistic or non-artistic. The emphasis shifts from the storage imperative towards transmission and circulation. It is in fact debatable whether image portals in the Internet represent an "archive" at all. Is YouTube an archive or rather anarchival?

A photography portal like Flickr is a repository (in archival terms). "The digital archive is by nature a database." [28] So-called social media platforms like Facebook, Youtube or Wikipedia represent rather searchable data banks than archives in its proper sense. Flickr rather a random collection than a well-structured archive, since it is user-generated, a generative archive. Its order depends on the accidental meta-dating (tagging) by the content-providers, not on any archival logic. Its archival logistics is rather the underlying algorithmic structure of image database management. Archives of photographic images themselves get in motion. [29]

In most cases, though, image contents can still not be algorithmically searched and accessed. The uncalculable is the real challenge to the "digital archive". Most photographic collections in the Web are rather libraries and not itself an archive. Rick Prelinger defines the Internet Archive in San Francisco itself as a "nonprofit digital library"; preservation is neither its mission nor its practice. [30] It is open access which distinguishes such a library (or musée imaginaire) from the archive which tends to keep secrecy by definition - like the protected mode within microprocessors. [31]

From semiotic analysis to "cultural analytics"

In so-called digital humanities, the archaeologists of knowledge are not exclusively human scholars any more but algorithmic media as well.

This is connected with a subtle shift from cultural (mostly semiotic) analysis of photography to "Cultural Analytics" (in terms of Lev Manovich), that is: computer-based matching. [32]

Here, statistical rather than semantic analysis of huge amounts of grabbed single photographies (of which one special form is films as dissected into single frames) performs digital image processing such as pattern analysis and subsequent two-dimensional re-visualisation. When such algorithmically calculated data re-appear by computer graphics on the screen, digital photography is transformed into visual diagramatics. A new kind of archival iconology thus arises, based on logical operations rather than content analysis.

The dominant criteria for the sorting of digital or digitised (sampled) photographies come from within their visual information, that is: digital pixel values such as hue (the color type or tone), saturation (a measure of how much it has been diluted), and lightness (intensity of light reflected from objects). [33] "In other words, we describe images with images." [34]

The Software Studies Initiative at the University of California in San Diego has developed a couple of such tools which are available online. All of the sudden, terms derived from statistical mechanics and physical thermodynamics enter which have been adopted for the mathematical theory of information by Claude Shannon; entropy becomes a measure for the accidental or improbable distribution of picture elements.

 Creative dis/order and the temporalisation of thephotographic archive

 Finally the photographic archive might become poietical itself, by generating new patterns of making use of stored visual evidence.

There is a reverse proportional memory economy at work with photographic archives. Physical storage of the photographic print provides, when being taken care of by professional conservation, a relatively stabile enduring memory, but more difficult to access. Once being digitised, the electronic image is open to almost real time access and new search options like similarity-based image retrieval; at the same time, the "virtual" essence of the electronic image becomes more fragile and subject to alteration than ever.

The traditional architecture (tektonics) of the archive is based on classificating records by inventories. This is being replaced in the digital media by order from fluctuation, that is: dynamic order. But this is an "archive" no more, but algorithmically ruled processuality.

In such a new order, images cannot only be retrieved as contained in their frames, but even by their atomic elements, pixelwise. Thus even what has not been meta-dated at all by human indexing can be automatically retrieved, opening new options of visual memory (be it in photography, be it in film). [35] Such a distribution of image elements does not belong to the library or the traditional archive any more, but builds up a new, mathematised generative principle, thus: an archive in the Foucauldean and Shannonean sense, being based on information itself. This new panopticism is being applied by commercial and military agencies already. New software like Microsoft's Photo DNA which allows for the automated idenficiation of - for example - child pornography on websites already indicates by its name that the basis of biological and technomathematical life forms start to converge.

The nostalgia for archival order is of course a phantasm surviving from the age of print. The alternative is a media culture dealing with the virtual an-archive of multi-media in a way beyond the conservative desire of reducing it to classificatory order again. Data trash is, positively, the future ground for media-anarchaeological excavations. [36]

Maybe, instead of thinking the archive in terms of order by classification, we have to think entropically, that is: allowing for a certain amount of disorder, which contains, according to communication theory, a higher measure of (possible) information. In a lecture under the title "The Storm-Cloud of the Ninetenth Century" Ruskin in 1884 implicitely replaced the museum like concept of classification by a theory of an archive in motion, a kind of steady-state. Instead of the order of things attributed to culture within the well-known Victorian museums, Ruskin founds in the weather a thermodynamic phenomenon which brings forces into play that radically alter ordinary mechanistic representation of nature: order by fluctuation, a form of order understood as process rather than state. Entropy - the conceptual enemy to the traditional archive as authority of tradition - thus is not just the negation of order but rather its alternative, "an organizing principle of disorder" [37] that all of the sudden makes sense when observed from on high. Such analysis oscillates between the micro- and the macrophysical level and results in cultural and even political aesthetics. Cloud modelling (developed for weather forecasting) is the name of the challenge to answer this anarchivic dynamics by fast calculation.

Maybe instead of thinking the archive in terms of order by classification, we have to think entropically, that is: allowing for the highest degree of disorder, which contains, in communication theory, the highest degree of (possible) information.

Archives used to be intended for keeping records virtually for eternity (or at least an approximative, asymptotic eternity as allegorized by the ancient Greek god Aion, different from Chronos and Kairos), like the time of nuclear and biological "half live" (Halbwertzeit) which can not be experienced by humans but it sublimely there.

The emphatic notions of past and future are being replaced by almost immediate retention and protention based on immediate intermediary memories - medium, the "inbetween" (Aristoteles´ notion of τo μεταξú) in a radically temporal sense, close to the temporal logic of electronic or neuronal circuits.

Just as digital art challenges "the conditions of archiving in our current regime of telecommunications" [38], photography from the beginning has not just been about permanent fixation of images but as well about immediate transmission; Alexander Bain already in 1844 invented a system for image telegraphy. With photography, the image not only became durable but as well in an antithetical way evanescent - a tendency enhanced by the very nature of the electronic image (fluxus in every sense), and in the age of digital media the image becomes coded information in a channel.

Unlike traditional image archives, Web based online collections of photographies are being updated almost by the minute. The radical temporalisation of image collections transforms the "archive" dramatically, with the recent "Web 3" economy leading to the real time net.

It is this dynamic dimension of the Web which is still largely beyond the scope of search engines which survey static web pages while relegating real time dynamics to the so-called deep web. "Thus archives still exist, helping you find your way around the anarchive of the net." [39]

The current search engines themselves are the real archives of the Internet. The whole Google architecture is reminiscent of an archive. But this is not the classical archive any more, but a processual one, with the Page Rank algorithm re-generating the ranking of retrieved information according to statistical and referential (URL links) values and weighting (the genotypical level). It is still a rule governed, programmed system which organises information so that it may be retrieved, but different to the traditional archive this archival "inventory" is updated - and indeed reconfigured - at an incredible speed: always another archive (on the phenotypical side).

Visual immediacy: Towards a dynamic technology of photographic image retrieval

 Most image extraction from analogue photographic archives is done by the grip on the single print as the storage medium, not by accessing its smallest picture elements. [40] In order to get a photography from a media library, one still has to type a verbal term into the search machine - even if the interface literally promises vision.

Is it possible to navigate through large amounts of images without being guided by verbal language? Is there something like an im-mediate access to images, unfiltered by words? The answer is in the new mode of image existence: the alphanumeric code, the symbolic regime of the digital image.

Expressing pictures by numbers undoes the old dichotomy between image and meta-data; there is rather an implosion of images and numbers in digital space.

In current image coding standards (MPEG7), the visual content and the meta-data are contained within the same file - a kind of mirror of the von-Neumann-architecture of computing itself.

The computability of images has been preceded by the art of Renaissance perspective (the rules of projective geometry), anticipating depiction by photographic means. Reversely (analytically) this makes possible to calculate pictures out of numbers and rules.

But should a digital photography still be called an "image"? What is a photography: a set of data, a format, an “epistemic thing“ (Jörg Rheinberger)? And at what moment does it become an image? By human perception only, or independent from human awareness already within its medium? Without human interpretation of certain visual patterns, the image would just be a cluster of data. Optical signals become information “in the eye of the beholder“ only. The computer can deal with the symbolical analysis of physical data only, not with the imaginary.

Michel Foucault's archaeological and archivological analyses autopoietically refer to the alphabet-based world of textual libraries. But "discourse analysis cannot be applied to sound archives or towers of film rolls." [41] What digital space allows for instead is the option of navigating images in their own medium - without changing from visual to verbal language at all. In digital space, the task of searching images does not only mean searching for images, but has a second, active meaning as well: images that search for similar images, without the interception of words - navigating images in Dataland (as named in 1973 by William Donelson), not in the Gutenberg Galaxy (McLuhan 1962).

Different from printed letters in a book, the symbols in digital technoscapes are arranged and distributed algorithmically.

Humans almost irresistably interface to photographic images in an iconologic way. Let us thus search for visual knowledge instead uncovered from within the visual endo-data: entering the image itself (data-immersion), which is the media-archaeological gaze that can be performed by machines of image processing bettern than by human perception. Such informatised organisation of visual knowledge generates diagrams (which is as well the Deleuzean intepretation of the Foucaultdean archive) - infomapping. Our visual culture is still dominated by semiotically iconic, photographic-like images; the twenty first century though allows for genuinely computer-generated visual information, closer to diagrams than to "images", which will eventually take their place and enable unprecedented types of "visual" representations.

There is a sublime knowledge already implicit, virtually “dormant“ within the electronic image, which - different from external inscriptions by meta-data - waits to be un-covered from within. A critique of external description of photographies draws on the assumption that there is a kind of visual knowledge which either does not need to be meta-dated or cannot be grasped by logocentristic addresses all. Even if the vocabulary applied in is user-generated (thus individualised, different from an archival thesaurus), it is still logo-centristic. Let the image rather be informative itself - by means of operating with values that are, already, intrinsic to the image in its digital state when the essence of the "image" itself transforms into something else: alphanumerical, bit-mapped data. For instance, computer-based retrieval can find all edges in a bit-mapped image. Such a “digital image“ is an image no more; what looks like images, is rather a mathematical function of data distributions.

Digital data-bank of images, when cleverly adressed, render a kind of knowledge to us which would otherwise be unimaginable in the Gutenberg printed information culture. That is, digital images render aspects of visual knowledge which only the medium knows, virtually in the „unconscious“ of the daten-bank. The media-archaeological program is to uncover such virtual visual knowledge.

Navigating images on the borderline of digital addressability

Any archival record, as opposed from being looked at individually, gets its meaning from a relational structure (which is the archival structure), the contingency to other documents. But opposed to the archival algorithms (its taxonomies) which operate on symbolic records within its own medum (the alphabet), a photographic archive is rather a collection of symptoms than an archive proper, due to the indexical nature of its records.

The digitisation of photographic archives now promises “that images that traditionally resisted the human attempts to describe them with precision – will be finally conquered“ (Lev Manovich) - now that images are being understood themselves as data sets, as clusters of pixels and colour values.

Addressing and sorting non-scriptural media remains an urgent challenge which, since the arrival of fast-processing computers, can be met by digitising analogue source material. The result is not necessarily better image quality but, rather, the unforeseenability to address not just images (by frames) but every single picture element (each pixel).

Images and sounds have become calculable and thus capable of being exposed to pattern-recognition algorithms. Such procedures will not only media-archaeologically "excavate" but as well generate unexpected optical statements and perspectives from an audio-visual archive that can, for the first time, organise itself not just according to meta-data but according to its proper criteria - visual memory in its own medium (endogenic).

Contrary to traditional semantic or iconological research in the history of ideas, such an endogenic visual archive will no longer list images and sequences according to their authors, subject, and time and space of recording. Instead, digital image data banks will allow visual sequences to be systematised according to genuinely iconic notions and mediatic rather than narrative common-places(topoi), revealing new insights into their im/material values. Our predominantly script-directed culture still lacks the competence of genuinely visual navigation.

 Mixed pixels of differnt colour may search for twin pixels by colour similarity. [42]

But for more sophisticated forms of visual rhetoric the computer is not yet capable; so far he can not really identify the whole of an object from the sight of a part of it. The computer in its traditional sense as logic machine is not brilliant in spotting associations between seemingly unrelated pieces of information and deriving generalisations of images, therefore fuzzy computer-sorting for useful comparisons of similar but not identical images on the basis of new protocols has been developed - just like neurons in the human brain do not primarily process, recall and transfer iconological content but rather patterns of memory. The image here exists rather in a structural, that is: archival latency.

The real iconic turn in adressing photographic images in archives is still to come - a visual sorting of images on the threshold of digital image processing and retrieval, which is digital image archaeology by functions like “searching images“. Instead of having to meta-date images by words, we can handle the data within the image itself. Photographic images and soundtracks can therefore be made accessible in their own medium, if only perfectly adequate algorithms of shape and pattern recognition are being made available.

(By statistical operations, evidence can be revealed which has never been seen before in images. Actually, the mathematician David Mumford has reduced the vocabulary of picture elements (would be pixels?) in Western visual culture down to 23 elements - just like the letters of the (Greek) alphabet.[43] Image-endogenic systems of sorting such as geometric topologies of images or even cinematographic sequences replace meta-dating. Whereas previous image sorting in a primarily writing-based culture has so far been clearly iconologically orientated (Erwin Panofsky), computing now offers the possibility of applying non-semantically operating image-sorting programs which rather recognizes formats and creates a strictly form-based image assortment, turning the ground (medium) into figure itself (an argument derived by Marshall McLuhan from Clement Greenberg's analysis of modernist painting).)

Visual search engines that can deal with "semantic" queries are still in their infancy - for example crawling the web for illegal trade-mark copying. Search & destroy: Let us not forget that “the similarity-based images retrieval technology is either militarily or commercially, not really culturally driven“ (Lev Manovich). In his film called Eye / Machine (, the film maker Harun Farocki draws our attention to operative images. So-called intelligent weapons as well become data-driven by matching images, not pre-directed by meta-data any more.

"Contentism" is the iconological heritage and cultural burden which hampers our digital performances. Beyond cultural software, is there something like non-cultural images? [44]

The metadata provided by an image database software to organise digital photo collections tell us all kinds of technical details such as what aperture my digital camera used to snap this or that image – but nothing about the image content from within (which is the media-archaeological perspective). Calculating images, MPEG-7 allows for layered image composites and discrete 3D computer generated spaces; this means a shift from low-level to high-level meta-data that describe the structure of a media composition or even their semantics.

For monitoring sytems to process a large amount of electronic images, such as human faces, automated image retrieval systems have to get rid of semantic notions of Gestalt. This is why the IBM Query By Image Content system does not radically decide in the dialectics between semantic versus non-semantic information, but rather distributes the task according to the respective strength in the human-machine interface. This instanciation of the dynamic image archive is a retrieval system for computer-based search for non-semantic aspects of a digital image (a mathematical operation), but can be supplemented by human help (tagging) for the semantic, iconological aspects.

"Humans are much better than computers at extracting semantic descriptions from pictures. Computers, however, are better than humans at measuring properties and retaining these in long-term memory. On of the guiding principles used by QBIC is to let computers do what they do best – quantifiable measurements – and let humans do what they do best – attaching semantic meaning".[45]

- which allows for the difference between human (neurological) and digital (algorithmic) data processing to take place, thus not trying to efface, but to creatively enhance the human-computer-difference where they meet. George Legrady's above mentioned installation Pockets full of Memories has been such a mixture of both human (semantic tagging) and inhuman (algorithmic) sorting of images. In his up-dated version called Cell Tango (, Legrady (together with Angus Forbes) displays a projection of constantly changing cellphone photos.[46] The photos are first sent by individuals to, and then projected rhythmically over a large, black screen in a variety of patterns. Fresh snapshops swiftly adjust to that mosaic according to formal criteria (image-based matching) and according to their tags (meta-data), mingling with photos taken from the photo-sharing Web portal Flickr. In one of the four modalities of the installation, "Cell_Bin", the most recent images are placed on the black screen first, and an algorithms randomly distributes them. The space left inbetween is successively filled by smaller incoming photographies. This loosely coupled patterns evolve dynamically. In this form of media art, algorithmic information is the artist's main medium.[47]



  1. Tjebbe van Tijen, ‘We no longer collect the Carrier but the Information’, interviewed by Geert Lovink, MediaMatic, 8:1, 1994 ("The Storage Mania Issue"). back
  2. This is of course a play with a quote in Walter Benjamin, ‘The Work of Art in the Age of Mechanical Reproduction’ (1936), Illuminations,London, 1973, 238. back
  3. Vilem Flusser, Für eine Philosophie der Fotographie, Göttingen, 1984. back
  4. Wolfgang Kemp, Theorie der Fotografie I (1839–1912),Munich, 1980, 121. back
  5. Ira M. Sage, ‘Making Machines Remember’, ProductEngineering, Bd. XXIV, April 1953, 141. back
  6. Vilém Flusser, DieSchrift.HatSchreibenZukunft?, Frankfurt/M, 1992. back
  7. A term borrowed from Elizabeth Skadden, Collapsing New Buildings, Master Thesis at the Rhode Island School of Design. See as well Wolfgang Ernst, ‘Archive, Storage, Entropy. Tempor(e)alities of Photography’, in Krzysztof Pijarski, ed, The Archive as Project. The Poetics and Politics of the (photo) Archive,Warsaw, 2011, 67-86. back
  8. Malin Wahlberg, ‘A Relative Timetable. Picturing time in the era of new media’, John Fullerton and Jan Olsson, eds, Allegories of Communication. Intermedial concern from cinema to the digital, Rom, 2004, 93-103. back
  9. Trond Lundemo, ‘In the Kingdomof Shadows. Cinematic Movement and Its Digital Ghost’, Pelle Snickars and Patrick Vonderau, eds, The YouTube Reader,Stockholm, 2008, 314-329 (316f). back
  10. Tomas Fryza, ‘A Complete Video Coding Chain Based on Multi-Dimensional Discrete Cosine Transform’, Radioengineering, 19:3, September 2010, 421-428 (421). back
  11. Adrian Mackenzie, ‘Codecs’, Matthew Fuller, ed, Software Studies. A Lexicon,Cambridge,Mass.,London, 2008, 48-55 (50f). back
  12. Subject of the international workshop The Anarchival Impulse in the Uses of the Image in Contemporary Art,Museum ofContemporary Art,Barcelona, October 24th, 2012. back
  13. W. E., Stefan Heidenreich, Ute Holl, eds, Suchbilder. Visuelle Kultur zwischen Algorithmen und Archiven,Berlin, 2003. back
  14. Lev Manovich, "Metadating" the Image, Lev Manovich, et al, Making Art of Databases, Rotterdam, 2003, 3. back
  15. ‘Sie <sc. Ähnlichkeitswahrnehmung> huscht vorbei, ist vielleicht wiederzugewinnen, aber kann nicht eigentlich wie andere Wahrnehmungen festgehalten werden’, Walter Benjamin, ‘Lehre vom Ähnlichen’, Walter Benjamin, Allegorien kultureller Erfahrung. Ausgewählte Schriften 1920-1940,Leipzig,1984, 125-130 (127). back
  16. See for example the installation BLOW_UP T.V. of Angela Bulloch in the gallerySchipper&Krome,Berlin, September to November 2000. back
  17. Wolfgang Ernst and Stefan Heidenreich, ‘Digitale Bildarchivierung: der Wölfflin-Kalkül’, Sigrid Schade and Christoph Tholen, eds, Konfigurationen. Zwischen Kunst und Medien,Munich, 1999, 306-320. back
  18. van Huisstede 1995: 158: ‘Wenn es jemals ein Projekt gegeben hat, das in einem elektronischen Medium wie der CD-ROM angemessen zu präsentierten wäre, dann ist es der Mnemosyne-Atlas’. back
  19. Claus Pias, Maschinen/lesbar. ‘Darstellung und Deutung mit Computern’, Matthias Bruhn, ed, Darstellung und Deutung. Abbilder der Kunstgeschichte,Weimar, 2000, 129. back
  20. Duncan Davies, Diana Bathurst u. Robin Bathurst, The Telling Image. The Changing Balance between Pictures and Words in a Technological Age,Oxford, 1990, 64f. back
  21. See; see as well back
  22. Bertolt Brecht, ‘Der Rundfunkt als Kommunikationsapparat’, Gesammelte Schriften, 18, Frankfurt/M, 1967, 117-134. back
  23. Timo Honkela and Juha Winter, Simulating Language Learning in Community of Agents Using Self-Organizing Maps, Helsinki University of Technology, Publications in Computer and Information Science, Report A71, December 15, 2003. back
  24. Vannevar Bush, ‘As We May Think’, Atlantic Monthly, July 1945; duchier/misc/vbush/vbush-all.shtml, 6 back
  25. Philippe-Alain Michaud, Aby Warburg et l’image en mouvement,Paris 1998. back
  26. The digital image sonification installation Voice of Sisyphus has been presented at the 18th International Conference on Auditory Display (ICAD-2012), June 18–22, 2012,Atlanta,USA. See back
  27. Michel de Certeau, ‘L´espace de l´archive ou la perversion du temps’, Traverses. Revue du Centre de Création Industrielle, 36, January 1986, 4-6. back
  28. Pelle Snickars, ‘The Archival Cloud’, Pelle Snickars and Vonderau, eds, 2009: 292-313 (304). back
  29. Ekekhard Knörer, ‘Trainingseffekte. Arbeiten mit YouTube und UbuWeb’, Zeitschrift für Medienwissenschaft, 5:2, 2011, 163-166. back
  30. Rick Prelinger, ‘The Appearance of Archives’, Pelle Snickars and Patrick Vonderau, eds, 2009: 268-274 (268). back
  31. Friedrich Kittler, ‘Protected Mode’, Manfred Faßler / Wulf Halbach (eds.), InszenierungenvonInformation.MotiveelektronischerOrdnung, Gießen, 1992, 82-92. back
  32. For a case study in Cultural Analytics, see Matthias Wannhoff, ‘Finden, was wir nicht suchen können’. Ein Versuch in algorithmischer Spielfilmanalyse mittels Cultural Analytics (summer 2012), (section "Hausarbeiten online"). back
  33. Oge Marques, Practical Image and Video Processing Using MATLAB,Hoboken, 2011, 398. back
  34. Lev Manovich, ‘How to Compare One Million Images?’, David M. Berry, ed, Understanding Digital Humanities,Basingstoke, 2012, 249-278 (263). back
  35. Harun Farocki, ‘Arbeiter verlassen die Fabrik’, Meteor – Texte zum Laufbild, 1, Dezember 1995, 49-55 (50). back
  36. Links to recycling: Redundant Technology Initiative and Mark Napier´s back
  37. Thomas Richards, ‘Archive and Entropy’, TheImperialArchive.KnowledgeandtheFantasyofEmpire,London andNewYork, 1993, 73-110 (86f). back
  38. Charlie Gere, New Media Art and the Gallery in the Digital Age; back
  39. TS Kjetil Jakobsen, ‘Anarchival society’, Eivind Røssaak, ed, The Archive in Motion. New Conceptions of the Archive in Contemporary Thought and New Media Practices, Oslo, 2010, 127-154, referring here to Alexander Halavais, Search Engine Society, Cambridge, 2010, 16. back
  40. It should be emphasized here that even if the analogue photochemical images is built up out of points and decomposes into points, this is not to be confused with the numerical picture element ("pixel"). back
  41. Friedrich Kittler, Gramophone - Film - Typrewriter,Palo Alto,Cal.,1999, 5. back
  42. See the Flash animation on top of the web site back
  43. See his Algebraic Geometry and his The red book on varieties and schemes (1999) back
  44. See Wolfgang Ernst, ‘Sehen wie ein Scanner’, forthcoming, Claus Pias, ed, Kulturfreie Bilder. Erfindungen der Voraussetzungslosigkeit, Berlin. back
  45. Myron Flickner, et al, ‘Query by Image and Video Content: The QBIC System’, Mark T. Maybury, ed, Intelligentmultimediainformationretrieval,Menlo Park,CA, 1997, 7-21 (8). back
  46. See back
  47. Fifield (Boston Cyberarts Inc.), ‘Can you see me now?’, The Boston Globe,; accessed August 2010. back