Semantic Web Workshop - ISNID

Inferential Semantics for Networked Image Data:

Supporting the Derivation and Management of Semantic Attributes for Web-Based Multimedia Content

Harold Boley (DFKI) and Jos De Roo (AGFA)

Background (AGFA and DFKI)

• At Agfa, the declarative Eve and Euler systems have been developed for capturing and transforming networked image data.
• The Euler work has incorporated image annotations on the basis of the W3C's RDF standard.
• At DFKI, the inference rule system RFML has been developed, which can be applied to annotations for deriving new semantic attributes.
• The joint RuleML Initiative has started to unify the various markups for XML and RDF inference rules.

Building the future (taken from Tim Berners-Lee)

Problem

• In practice, there is a lot of simple rules that can operate on a combination of extracted image features (obtained via image analysis) and additional image-related data.
• The implicit meaning in image data can thus be made explicit.
• We study the issue of how to infer (only) meaningful facts from networked image data.
• In the non-distributed case, the semantics of some centralized image data can be regarded model-theoretically, as the set of facts whose deduction is sanctioned by the given rules.
• For example, rules can infer the transitive closure of a centralized 'connected' attribute. In the distributed case, we usually have an open network of image data, for which no model-theoretic semantics exists.
• In the example, the transitive closure of a distributed multi-sensor-derived 'connected' attribute may never form a single stable set.

Proposal

• We thus propose tailored, "least power" mechanisms for inferencing with attributes specific for images such as topology, gemometry, texture and color.
• These mechanisms will be layered such that the appropriate mechanism is easily identified for each given task (e.g., a functional mechanism will be used for computing with measurements, thus avoiding relational search whenever possible).
• The work will be done on the basis of real-world (Agfa) image examples and their annotations. The examples will be classified along relevant dimensions, e.g. according to descriptive complexity, for testing the layered inference mechanisms.

Architecture

• Furthermore, to exploit the networked character of the given (Agfa) image data, we will study a "Webized" architecture for distributing inference mechanisms over appropriate subsets of input data, and for channelling output data to possible further processing steps.
• For this, metadata content and capability descriptions of, respectively, data and mechanisms will be explored.
• We envisage an efficient, platform-independent Java implementation of the developing inferential semantics for networked image data.