Open access

Anticipative Generation and In-Situ Adaptation of Maneuvering Affordance in a Naturally Complex Scene

Written By

Kohji Kamejima

Published: 01 December 2009

DOI: 10.5772/6841

From the Edited Volume

Advances in Human-Robot Interaction

Edited by Vladimir A. Kulyukin

Chapter metrics overview

2,109 Chapter Downloads

View Full Metrics

1. Introduction

Computational resources combined with advanced mechanical systems rapidly expand the scope of ‘informatic vicinity’ (Kamejima, 2006) in which machine perception is delegated and networked to support human’s situation understanding and decision making. For instance, student knowledge can be expanded by space craft operated interactively from the classroom (Coppin et al., 2000). Final decisions on social safety in large scale natural disasters are determined mainly based upon information gathering and damage evaluation through network systems (Hamada & Fujie, 2001). Recent computer controlled vehicles, in particular, are developing the capability of understanding the situation for supporting human’s inherent maneuverability (Özgner & Stiller, 2007).

By expanding the scope of perception to satellite-roadway-vehicle network (Kamejima, 2008), we have an implementation of the informatic vicinity as shown in Figure 1. The latest Earth observation systems can provide a bird’s eye view for vehicles to identify themselves as consistent parts of the real world; current vision systems can analyze complex images captured at scene A to control a vehicle mechanism along the roadway specified in one frame of a specific satellite image (Urmson et al., 2009); and by using Global Positioning System (GPS), the vehicle can be localized within a small area in the satellite image in which the roadway pattern at scene A can be matched exactly and extended towards a possible goal, e.g. Scene B, prior to physical access (Kamejima, 2007). Therefore, the informatic vicinity can be exploited as a technological basis for the anticipative road following scheme. In addition, through the informatic vicinity, vehicles can mutually provide critical information for analyzing not-yet-accessible scenes, for example, precautionary safety information to human drivers and make effective predictions for machine vision as well.

As a consequence of evolution in the real world filled by uproarious illumination and reflection (Parker, 2003), the range of human’s perception is restricted to the physical-geometric perspective from a specific point of view. To facilitate human’s inherent perception in complex situations, the view from the vehicle should be located and oriented anticipatively in the informatic vicinity covering all possible transitions from the current scene. Such a cooperative system is expected to act as a substitute for essential parts of humans cognitive ability (Kamejima, 2008). As for playing an essential role in cooperative systems, however, the perceptive delegation is required to maintain on-going conformability

Figure 1.

Anticipative Decision Making in Informatic Vicinity.

with human capacity under the schematics of serious contradiction. In other words, subsequent maneuvering processes are anticipatively activated towards scenes beyond the horizon of human’s perception.


2. Existence of perceptual invariance

In a naturally complex scene, we are surrounded by optical flux modulating complete information that can afford to induce spontaneous maneuvering processes preestablishingly. For anticipative decision making, such affordance should be captured and transferred beyond the horizon of perception via a reconfiguration process; the a priori orientation of the roadway specified in the bird’s eye view should be mapped to the scene image via a posteriori adaptation of maneuvering processes. To this end, we introduce a fractal based representation of a roadway pattern in a naturally complex scene.

Figure 2.

Multi-Aspect Approach to Complex Scene.

Following the collage theorem within the context of multi-fractal modeling (Barnsley, 2006), the expansion of the open space can be associated with the same class of self-similar patterns in multi-viewpoint imagery as illustrated in Figure 2. An approximation of roadway segment can be generated in satellite imagery as a ‘carpet’ (right aspect); this segment is mapped from the satellite image to associated scene images to yield a skewed ‘gasket’ (left aspect); through the multi-aspect access to a roadway, the perception process induces anticipative information into a part of the scene called the horizon of intension. This implies that fractal representation of a roadway pattern is transferable through the informatic vicinity as a common basis for mutual mediation of perceptive delegates. In what follows, the fractal coding scheme is introduced on an ineluctable information: noisy pattern covering really existing objects. By combining the randomness, we have robust fractal code for anticipative generation and in-situ adaptation of the maneuvering affordance.


3. Randomness-based approach

Suppose that the roadway is segmented in the bird’s eye view in terms of a sequence of vectors { v } to be ‘downloaded’ in a scene image. With this anticipative segmentation, we can induce an aspect to be associated with the horizon of intention as shown in Fig.3 where the scope of perception is structured in the scene image in a stochastic sense; the belief on the open space is supported by a probability distribution ρ confined by the estimate of the boundary; within the horizon of control, the maneuvering process is guided to follow the boundaries under human’s spontaneous decision making; and, generated path is extended towards the scope of perception as a stochastic process susceptive to not-yet-encountered uncertainties. Assume that the image of segment v is transferred via inter-viewpoint association and consider anticipative decision making problems in the information structure: how to project v in the observed scene; how to evaluate the depth and width of the open space; and, how to adapt the maneuvering process anticipatively to the open space.

Figure 3.

Information Structure for Anticipative Decision.

Let be the projection of the vector v into the image plane = X × Y. From the viewpoint of ecological optics, a priori basis for spontaneous mobility is provided by random texture mapped onto an image plane . Let f() be the brightness at the pixel and suppose that views of the objects to be detected are uniquely represented as subsets of and identified within : the Borel field generated by the subsets of . In the totality of object images, then, we can introduce basic probability space (, , P) where P() is a uniform probability measure assigned to pixels ω without any a priori knowledge. By definition, P() indexes a priori probability for the pixel ω to be a part of an object.The a priori information is transferred to the observation f() via complex physical, mathematical and cognitive processes. As for the physical complexity, we can utilize various results on geometric-chromatic investigations on optical information transmission. On the other hand, the mathematical complexity for assigning the pixel to a specific part of target object yields combinatorial explosion. However, without serious loss of generality, the combination of the existence theorem (Hutchinson, 1981) with ‘collage’ theorem (Barnsley et al., 1986) resolves the computational difficulty into the following non-deterministic representation:

where Ξ is a fractal version of the object image; denotes a fixed set of contraction mapping of size :

In this representation, an attractor point Ξ can be mapped finally to the entire pattern Ξ through a finite sequence of possible selection μi . By substituting the iteration of random selection μi in a fixed set ν for pattern association (1), we can identify constrained aggregation Ξ with a trajectory of 2D stochastic processes driven by . Due to the ‘whiteness’ in random selection μi , the expansion of the attractor Ξ of geometric singularity can be visualized as a distribution satisfying the following self-similarity
with ‘coloring probability’ { pμi } for selecting μi in fixed set ν. The existence of the invariant measure implies the possibility for image based association f ~.Being given the invariant measure (2), we can evaluate the probability () for capturing unknown fractal attractor Ξ as the solution to the following equation (Kamejima, 2001):

where ρ = log2 denotes the complexity index of the imaging process (1) with (2). By this implicit dependence, the capturing probability () can be generated prior to the specification of the mappings μi .

Fig. 4. Detection of Fractal Model in Noisy Background.Define the stochastic feature of fractal attractor Ξ as the totality of local maxima of the capturing probability (), i.e.,

On this discrete set, we can verify the consistency of the mapping set through the following computational test:


By definition, the following random process

expands a point θ towards the fractal attractor Ξ successively. Therefore, the connectedness of the point set can be indexed in terms of the distribution of the invariant subset within. This provides a computational basis for the detection and connectedness analysis of the fractal model . Figures 4 and 5 demonstrate the detectability of fractal models. In these figures, noisy observations of fractal attractors (a) are identified with the measure to generate the capturing probability () visualized as (b); from the stochastic feature detected via local analysis of smooth distribution (), the invariant feature is extracted as closed links (c); the existence of the invariant feature Θ activates the 2D random process (6) to visualize connected spaces (d). Therefore, we can restore known fractal models corrupted in noisy observation.

Figure 5.

Detection of a Fractal Model in Noisy Background.

Considering the generic rule for associating the not-yet-identified invariant set with observed distribution f(), let a scene indicated in Fig.6 be observed by a viewer with the intentional coordinate system illustrated in Fig.7. By applying a 2D Laplacian filter to the scene, we can extract the noise component which can be utilized as a robust feature of the roadway image. Through the perspective projection, the power spectrum evaluated at the baseline of Δf-image (Fig.8) is expanded in accordance with the line shift towards the vanishing point as shown in Fig.9. Therefore, we can generate a generic representation of an open space in terms of scale information supported by the micro-structure of a roadway area.Following a multi-scale approach (Jones & Taylor, 1994), the brightness distribution f() can be decomposed into a function space consisting of class F = { fσ, σ > 0 } given by

Figure 6.

Shopping Street.

Figure 7.

Intentional Coordinate System.


where gσ is 2D Gaussian distribution with variance parameter σ; b stands for the aggregation of low frequency components such as ΔbΔgσ. By identifying the measure with an aggregation of ‘point images’, we can extract component images with specific scales as shown in Figure 10, where a set of Δgσ-filter with 2D transfer function (see Figure 11) is applied to the point image concentrated at the origin. Noticing the similarity of Δgσ and gσ filters near the origin, we have the following local estimates for the scale information at (Kamejima, 2005):


Let σ0 be the maximal scale associated with the noise component and define d as the depth parameter indexed along the ‘direction of intention’: the perspective projection of the orientation vector v on the scene image. Here we have the following representation of the ‘generic’ roadway model M(Ωd,



Figure 8.

Power Spectrum of Δf at the Base Line.

Figure 9.

Directional Fourier Transform of Δf.

Figure 10.

Δgσ Filtering of a Point Image.

Figure 11.

Power Spectrum of Δgσ Filter

In (9a), the expansion of the roadway is identified with a pattern Ξ  in which scale information is regulated by the linear diminishing rule from the bottom of the maneuvering affordance d0 to the vanishing point d. Hence, we can conclude that the scope of perception is confined in a probabilistic sense (9b) where the estimated distribution can be utilized as a ‘noisy’ observation of the self-similarity (1).

4. Fractal coding of perceptual invariance

Via the self-similarity process (1), an attractor point ξ is allocated to satisfy the following constraint with respect to the not-yet-identified contraction mapping μi ν:

where denotes the fixed point of μi. By evaluating the attractive force within the framework of the Hausdorff topology (Kamejima, 1999), we can introduce a simultaneous estimation scheme for model parameters (d,) with non-unique estimates of fixed points associated with not-yet-identified contraction mapping μj. To apply the articulation scheme (10) with non-deterministic kinetics, first, a pixel is associated with not-yet-identified attractor Ξ in a stochastic sense. Once we have observed the invariant measure, we can evaluate the probability () for capturing unknown fractal attractor Ξ as the solution to the following equation:
12Δφ(ωv )+p[Ξp-φ(ωv )]=0.E14

Following this, the image plane is partitioned in accordance with the fractal attractor to be detected. Since various types of attractors are simultaneously observed as object images (Barnsley, 2006) in practical imagery, generated information () is expanded to cover noisy patterns as well. To confine the distribution into a target attractor, let the initial guess for the fixed points Ωf={ωμif}be given as a perspective of the segment v and consider the articulation →{ Λi } as illustrated in Figure 12:


with statistical moments (ω_i,i) conditioned by :


where Ci denotes the normalization constant. In this articulation, the expansion of the domains Λi is indexed in terms of the following ‘Laplacian-Gaussian basin’:

In such a basin, we have the following circumscribing polygon within the context of statistical clustering:
where is the contact point with; and are unit vectors associating the fixed point with and, respectively; R denotes the rotation matrix. By adjusting to the boundary of the Laplacian-Gaussian basin (12) along the external normal vector, we have the following adaptation scheme of the fixed point estimate:

Figure 12.

Laplacian-Gaussian Basin.

In this scheme, the fixed points {} are mutually separated by the expansion of the Laplacian-Gaussian basins (12); on the other hand, the expansion of the fractal attractor to be generated is confined in terms of the contact points {}. As a result of this antagonistic dynamics, the update d are coordinated via the integration rule:
The statistical clustering is followed by geometric design and computational verification of mapping set. The self-similarity indicated in Fig.2 combined with the fixed point assignment yields the following description:
The consistency of fixed point allocation should be verified by the self-similarity analysis of the mapping set on the fractal attractor to be detected. To this end, we introduce the following computational test on a stochastic representation of the not-yet-identified Ξ (Kamejima, 2001):Proposition 1 Let be an invariant measure with respect to the mapping set. Suppose that is extracted from the capturing probability () associated with. Then there exists the invariant feature Θ satisfying the following constraint
The existence of invariant features Θ implies that the range of designed imaging process can generate a version of a fractal attractor indicating a connected open space in the roadway area. The combination of equations (9), (11), (12), (14), (17) provides a computational basis for the coding of self-similarity of complex random patterns.

5. In-situ adaptation via ground-object separation

By identifying the vanishing point = (d,) with a fixed point estimate the generic model (9) induces a geometric structure into the scene image as shown in Fig.13. A pixel in a Laplacian-Gaussian basin is non-deterministically attracted to one of the fixed points in due to the generativity of the self-similarity process. Despite non-deterministic allocation, the structural consistency of the set is verified by the existence of the capturing probability () supporting invariant subset Θ. Let ξ be the nearest point to the estimate of in the invariant subset Θ. By using the point ξ, we can specify the horizon of control as well as the depth of the boundary information (bL, bR) to be marked in the scene image. Therefore, the generic model (9) combined with fractal coding yields an estimate of the roadway area prior to object identification.Furthermore, we can design another generic model on the scale space information (8) to detect something perpendicular to the roadway. For this purpose, the mismatch with the generic model (9) is evaluated in terms of the following measure:

where denotes a pixel selected in the domain confined by the boundary information (bL, bR) and ξ; = (x, y+dy) is the upward extension of the pixel with the vertical interval dy. This pixel wise evaluation is chained to visualize not-yet-identified objects as follows:


where <> denotes the vertical chain of pixels with bottom to be grounded on the maneuvering affordance. The first term of this evaluation indicates the length of the vertical chain; and the second term indexes the probability for the chain to ground somewhere in a plane supporting the roadway area. In equation (19), the probability for the segment <> to be a part of the object is evaluated as the ‘breakdown’ of the generic model to induce linear scale shift in the scene image.

As shown in equations (9) and (18), roadway area and object images are separated as generic models based on as-is information. σωNoting that the connectedness of the detected roadway area is verified as the existence of a fractal attractor, we can utilize the aggregation

Figure 13.

Fractal Coding of Laplacian-Gaussian Basin

of grounding pixels as an estimate of the boundary of the roadway area confined by a distribution of ‘obstacles’ to be analyzed in the scene. Hence, we have the following computational scheme for object-ground separation:

  1. Fractal structure of maneuvering affordance is extracted in terms of 2D allocation of fixed points Ωf = {


  2. The imaging mechanism of the randomness distribution xΞpis designed in terms of mapping set v through simultaneous estimation of discrete information Ωf and induced 2D field ().

  3. The integrity of decentralized estimate Ωfis visualized as associated attractor Ξ generated by using the imaging process parameter v on the brightness distribution f().

  4. Self-similarity of the imaging process parameter v based on the estimate Ωdis verified via computational generation of invariant subset Θ on the local maximaθ: a pattern sensitive sampling of smooth field ().

  5. As a stochastic basis of the maneuvering affordance, the scale information provides complementary information; noisy observation of an open space xΞp with the distribution of the breakdown p().

  6. The boundary of maneuvering affordance can be detected through the aggregation of as the breakdown as 1D imagery <>.

The mapping set v can be designed non-uniquely on the estimate of fixed points. Such non-unique representation provides computational basis for decentralized perception. The geometry of the maneuvering affordance is transferable via multi-viewpoint imagery and reconfigurable through dynamic interaction with the scene.

The mechanism for the multi-viewpoint integration is illustrated in Figure 14; based on a priori information v in the bird’s eye view, the direction of the roadway Ωdis transferred to the scene image to specify an initial guess of fixed pointsΩd; by articulating the capturing probability () into the Laplacian-Gaussian basin { Λiϑ}

, a fractal model is designed in terms of mapping set v

; the fractal model is verified via computational detection of invariant feature Θ and adapted to the as-is distribution of boundary objects via ground-object separation.

Figure 14.

Adaptive Fractal Code


6. Experiments

Let a fixed point of the Gasket be associated with the vanishing point with a depth parameter d and suppose that the rest of the fixed points are allocated at both bottoms of the roadway image where maximum scale of random patterns max maybe detected. By introducing this initial guess, the equations (8) – (17) can be solved via the iteration process:

Steady state of the iteration process applied on a scene image is indicated in Figure 15 – 18. As an initial guess for starting the iteration (20), the set {ωμif} is assigned in a scene image displayed in Figure 15; three fixed points are allocated at top-center, left- and right bottom for specifying upper vertex and left-right vertexes of a ‘gasket’ pattern; as the result of the iteration (20), a mapping set with three fixed points {ωμif} is designed as a generator of a priori gasket model. The a priori fractal model based on the fixed point estimates generates the fractal attractor indicated in Figure 16. In this case, the scale of distributed randomness is limited by σ0 = 2 · σmin. The mapping set vassociated with the gasket model is verified via finite self-similarity analysis as indicated in Figure 17; the consistency of the fractal attractor to be generated is computationally tested via the generation of an invariant subset Θ and indicated as a closed link on a representation of ‘most probable’ attractor points.θ By the existence of Θ, the consistency of the measure xΞpwith the randomness distribution is verified as well as the self-similarity of associated attractor Ξ visualized in Figure 18. Thus, the estimate of the vanishing point (d,Ωd) is verified to be consistent with the generic model (9).

In complex scenes where the maneuvering area is clearly indicated as a lane mark as shown in Fig.18, designed mapping set specifies the boundary of the fractal model (bL, bR). Such boundary information is critical in the road following processes. In many practical situations, the boundary is obscured by sign patterns and occluded by obstacles as indicated in Fig.6. In such a scene, we can define the boundary of open space via ground-object separation. To this end, first, the scale shift of distributed randomness was matched with the generic model (9) to design a version of fractal code. The designed code was verified via a computational consistency test and visualized as shown in Fig.19. By the existence of the

Figure 15.

Roadway Scene to be Analyzed

Figure 16.

Fractal Coding

fractal attractor, the validity of the designed version of fractal code was verified as well as the perceptual consistency of the generic model. Hence, we can activate the ground-object separation process; the generic model (9) was applied to entire the scene image; the pixels of inconsistent scale estimate () were extracted and chained in the scene image as shown in Fig.20. As shown in this figure, resultant chains can separate the image of something perpendicular to the open space supporting the generic rule: the linear scale shift due to perspective projection. Thus, we can define a version of an effective boundary as the vertical chain of the breakdown points with the length over the noise scale: <> ≥ min. To confine the fractal model ν within the open space, we re-assign the fixed points = { } and re-activate the design process. The obtained fractal model was visualized in the scene image as shown in Fig.21. This figure demonstrates that the fractal coding of maneuvering

Figure 17.

Computational Verification

Figure 18.

Road Following Process

Figure 19.

Associated Fractal Attractor

Figure 20.

Object Separation Based on <&#media/image32.jpeg#11;9.5>-Model

Figure 21.

Fractal Sampling

affordance with breakdown detection yields plausible reference for the visual guidance along a perceptually boundary in a naturally complex scene.

Through these experimental studies, it was demonstrated that the anticipative road following results in the bird’s eye view can be applied to an extended class of roadway scenes as an a priori model. This implies that the design-and-refine steps of fractal coding can be applied to scenes consisting of objects covered by scale and chromatic randomness. This condition is satisfied in naturally complex scenes consisting of worn-out objects on which microscopic damage is expected to be uniformly distributed.


7. Concluding remarks

Fractal representation of the maneuvering affordance has been introduced on the randomness ineluctably distributed in naturally complex scenes. Scale shift of random patterns was extracted from scene image and matched to the a priori direction of a roadway. Based on scale space analysis, the probability of capturing not-yet-identified fractal attractors is generated within the roadway pattern to be detected. Such an in-situ design process has been demonstrated to yield anticipative models for road following process. The randomness-based approach yields a design framework for machine perception sharing man-readable information, i.e., natural complexity of textures and chromatic distributions. This implies that the fractal roadway model can be used for mental load reduction through human-machine cooperation.


  1. 1. BarnsleyM. F.2006SuperfractalsCambridge.UniversityPress.CambridgeU. U.K.,
  2. 2. BarnsleyM. F.ErvinV.HardinD.LancasterJ. 1986Solution of an inverse problem for fractals and other sets. Proceedings of the National Academy of Science, USA, 7 83 19751977
  3. 3. CoppinP. W.PellR.WagnerM.HayesJ. R.J.LiL.HallL.FischerK.HirschfieldD.WhittakerW.2000EventScope.Amplifyinghuman.knowledgeexperiencevia.intelligentrobotic.systemsinformationinteraction. In Proceedings of 9 9th IEEE International Workshop on Robot and Human Interaction (RoMan2000), 292296 , Osaka, Japan,. IEEE.
  4. 4. HamadaT.FujieM. 2001Robotics for social safety. Advanced Robotics, 3 15 383387
  5. 5. HutchinsonJ. E. 1981Fractals and self similarity. Indiana University Mathematical Journal, 30 713747
  6. 6. JonesA. G.TaylorC. J. 1994Solving inverse problems in computer vision by scale space reconstruction. In Proceedings of IAPR Workshop on Machine Vision Applications (MVA’94), 401404 401404 Kawasaki, Japan,. IAPR.
  7. 7. KamejimaK. 2001Generic representation of self-similarity via structure sensitive sampling of noisy imagery. Electronic Notes in Theoretical Computer Science, 46:, 20 20
  8. 8. KamejimaK. 2005Laplacian-gaussian sub-correlation analysis for scale space imaging. International Journal of Innovative Computing, Information and Control, 1 3 381399 .
  9. 9. KamejimaK. 2006Image-based satellite-roadway-vehicle integration for informatic vicinity generation. In Proceedings of 15 15th IEEE International Symposium on Robot and Human Interaction (RoMan2006), 334339 . IEEE,.
  10. 10. KamejimaK. 2007Randomness-based scale-chromatic image analysis for interactive mapping on satellite-roadway-vehicle network. Journal of Systemics, Cybernetics and Informatics, 4 5 7886
  11. 11. KamejimaK. 2008Chromatic information adaptation for complexity-based integration of multi-viewpoint imagery- a new approach to cooperative perception in naturally complex scene- International Journal of Innovative Computing, Information and Control, 1 4 109126
  12. 12. KamejimaK. 1999Nondeterministic kinetics associated with self-similarity processes with applications to autonomous fractal pattern clustering. In Proceedings 1999 1999 IEEE International Conference on Systems, Man and Cybernetics (SMC’99), pages VI:890895 , Tokyo, Japan,. IEEE.
  13. 13. ÖzgnerÜ.StillerC.2007 Systems for safety and autonomous behavior in cars: The DARPA grand challange experience. Proceedings of the IEEE, 2 95 397411
  14. 14. ParkerA. 2003In the Blink of an Eye. The Free Press, London, U. K.,
  15. 15. UrmsonC.BakerC.DolanJ.RybskiP.SaleskyB.WhittakerW.FergusonD.DarmsM. 2009Autonomous driving in traffic: Boss and the urban challenge. AI Magazine, 2 30 1728

Written By

Kohji Kamejima

Published: 01 December 2009