Identifiability of Quantized Linear Systems

1. Introduction

On-line parameter identification is a key problem of adaptive control and also an important part of self-tuning regulator (STR) (Astrom & Wittenmark, 1994). In the widely equipped large-scale systems, distributed systems and remote systems, the plants, controllers, actuators and sensors are connected by communication channels which possess only finite communication capability due to, e.g., data loss, bandwidth constraint, and access constraint. From a heuristic analysis perspective, the existence of communication constraints has the effect of complicating what are otherwise well-understood control problems, including the traditional methods, such as the $${\text{H}}_{\infty }$$control (Fu & Xie, 2005), and even the basic theoretic notions, such as the stability (De Persis & Mazenc, 2010).

Due to the constraints of the communication channels, it is difficult to transmit data with infinite precision. Quantization is an effective way of reducing the use of transmission resource, and then meeting the bandwidth constraint of the communication channels. However, quantization is a lossy compression method, and hence the performance of parameter identification, even the validity or effectiveness of identification may be changed by quantization, along with which the performance of adaptive control may deteriorate. This has attracted plenty of works. The problem of system identification with quantized observation was investigated in (Wang, et al, 2003, 2008, 2010), where issues of optimal identification errors, time complexity, optimal input design, and the impact of disturbances and unmodeled dynamics on identification accuracy and complexity are included.

In the light of the fundamental effect of quantization on system identification, it is necessary to pay attention to the parameter identifiability property of quantized systems. The concept of identifiability has been defined by maximal information criterion in (Durgaryan & Pashchenko, 2001): the system is parameter identifiable by maximal information criterion, if the mutual information between actual output and model output is greater than zero. However, the concept of identifiability in (Durgaryan & Pashchenko, 2001) is defined in principle, based on which there is no practical result. Reference (Zhang, 2003; Zhang & Sun, 1996; Baram & Kailath, 1988) have discussed the problem of states estimability, which is related closely with parameter identifiability, for that input-output description of linear systems with Gauss-Markov parameters can be transformed to state space model, and then the problem of parameter identifiability can be treated as state estimability. Reference (Zhang, 2003) has proposed the definition of parameter identifiability under the criterion of minimum maximum error entropy (MMSE) estimation referring to the definition of states estimability, and also obtained some useful conclusions. Reference (Wang & Zhang, 2011) has studied the parameter identifiability of linear systems under access constraints. However, there is few work on that for quantized systems.

This paper mainly analyzes the parameter identifiability of quantized linear systems with Gauss-Markov parameters from information theoretic point of view. The definition of parameter identifiability proposed in (Zhang, 2003) is reviewed: the linear system with Gauss-Markov parameters is identifiable, if and only if the mutual information between the actual value and estimates of parameters is greater than zero, which is extended to quantized systems by considering the intrinsic property of the system. Then the parameter identifiability of linear systems with quantized outputs is analyzed and the criterion of parameter identifiability is obtained based on the measure of mutual information. Furthermore, the convergence property of the quantized parameter identifiability Gramian is analyzed.

The rest of the paper is organized as follows: In section 2, we introduce the model that we are interested in; Section 3 discusses the existing definition of parameter identifiability, proposes our new definition, and gives analytic conclusion focusing on quantized linear systems with Gauss-Markov parameters; The convergence property of Gramian matrix of parameter identifiability for quantized systems is discussed in section 4; Section 5 and 6 are illustrative simulation and conclusion, respectively.

2. System description

Consider the following SISO linear system expressed by Auto-Regressive and Moving Average Model (ARMA) (Astrom & Wittenmark, 1994):

$$\begin{array}{l}y(k)+a_1(k)y(k-1)+\cdots +a_n(k)y(k-n)\\ =b_1(k)u(k-1)+\cdots +b_m(k)u(k-m)+e(k)\end{array}$$E1

where $\text{u}\text{(}\text{k}\text{)}$and $\text{y}\text{(}\text{k}\text{)}$are input and output of the system respectively, stochastic noise sequence$$\text{{}\text{e}\text{(}\text{k}\text{)}}$$is Gaussian and white with zero-mean and covariance R, and uncorrelated with$\text{y}\text{(}\text{k}\text{)}$, $\text{y}\text{(}\text{k}\text{-1)}\cdots$and$\text{u}\text{(}\text{k}\text{)}$,$\text{u}\text{(}\text{k}\text{-1)}\cdots$. Let

$$\text{θ}\text{(}\text{k}\text{)=[}{\text{b}}_{\text{1}}\text{(}\text{k}\text{)}\cdots {\text{b}}_{\text{m}}\text{(}\text{k}\text{) }{\text{a}}_{\text{1}}\text{(}\text{k}\text{)}\cdots {\text{a}}_{\text{n}}\text{(}\text{k}\text{)}{\text{]}}^{\text{T}}$$E2

be the parameter vector, where ${(\cdot )}^{\text{T}}$ denotes the operation of transposition, and let

$$\text{F}\text{(}\text{k}\text{)=[}\text{u}\text{(}\text{k}\text{-1)}\cdots \text{u}\text{(}\text{k}\text{-}\text{m}\text{) -}\text{y}\text{(}\text{k}\text{-1)}\cdots \text{-}\text{y}\text{(}\text{k}\text{-}\text{n}\text{)}{\text{]}}^{\text{T}}$$E3

then system (1) can be described as

$$\text{y}\text{(}\text{k}\text{)=}{\text{F}}^{\text{T}}\text{(}\text{k}\text{)}\text{θ}\text{(}\text{k}\text{)+}\text{e}\text{(}\text{k}\text{)}$$E4

Suppose that the parameter$\text{θ}\text{(}\text{k}\text{)}$can be modeled by a Gauss-Markov process, i.e.,

$$\text{θ}\text{(}\text{k}\text{+1)=}\text{Aθ}\text{(}\text{k}\text{)+}\text{Bw}\text{(}\text{k}\text{)}$$E5

(5)

where A, B are known matrices with appropriate dimensions; noise sequence $\text{{}\text{w}\text{(}\text{k}\text{)}}$ is Gaussian and white with zero-mean and covariance Q; initial value of the parameter $\text{θ}\text{(0)}$ is Gaussian with mean $\text{<mover accent="true">θ <mo>¯</mo></mover>}$ and covariance$\text{Π}\text{(0)}$. Suppose that$$\text{e}\text{(}\text{k}\text{)}$$, $\text{w}\text{(}\text{k}\text{)}$and $\text{θ}\text{(0)}$ are mutually uncorrelated. Hence, linear system (1) with Gauss-Markov parameters can be described by (4) and (5), i.e.,

$$\{\begin{array}{l}\text{θ}\text{(}\text{k}\text{+1)=}\text{Aθ}\text{(}\text{k}\text{)+}\text{Bw}\text{(}\text{k}\text{)}\\ \text{y}\text{(}\text{k}\text{)=}{\text{F}}^{\text{T}}\text{(}\text{k}\text{)}\text{θ}\text{(}\text{k}\text{)+}\text{e}\text{(}\text{k}\text{)}\end{array}$$E6

Due to bandwidth constraint of the channel, quantization is required. The discussion in the present paper does not focus on a special quantizer, but on general N-level quantization (Curry 1970; Gray and Neuhoff 1998) which can be described as:

$${\text{y}}_{\text{q}}\text{(}\text{k}\text{)=}\text{Q}\text{(}\text{y}\text{(}\text{k}\text{))=}{\text{z}}_l\text{,}\text{for }\text{y}\text{(}\text{k}\text{)}\in {\text{δ}}_{\text{l}}\text{,}\text{l}\text{=1,2,}\cdots \text{,}\text{N}$$E7

where $$\text{Q}\text{(}\cdot \text{)}$$is the general quantizer, $${\text{y}}_{\text{q}}\text{(}\text{k}\text{)}\in \text{{}{\text{z}}_{\text{1}}\text{,}{\text{z}}_{\text{2}}\text{,}\cdots \text{,}{\text{z}}_{\text{N}}\text{}}$$denote the quantizer outputs with $${\text{z}}_{\text{l}}\text{, }\text{l}\text{=1,}\cdots \text{,}\text{N}$$as the reproduction values; $${\text{δ}}_{\text{l}}\text{=(}{\text{a}}_{\text{l}}\text{, }{\text{a}}_{\text{l+1}}\text{],}\text{l}\text{=1,2,}\cdots \text{,}\text{N}$$denote the quantization intervals, where$${\left\{{\text{a}}_{\text{i}}\right\}}_{\text{i}\text{=1}}^{\text{N}\text{+1}}$$, $$\text{-}\infty \text{=}{\text{a}}_{\text{1}}\text{<}{\text{a}}_{\text{2}}\text{<}\cdots \text{<}{\text{a}}_{\text{N}\text{+1}}\text{=+}\infty$$are the thresholds of the quantizer.

The channel is assumed to be lossless. $${\text{y}}_{\text{q}}\text{(}\text{k}\text{)}\in \text{{}{\text{z}}_{\text{1}}\text{,}{\text{z}}_{\text{2}}\text{,}\cdots \text{,}{\text{z}}_{\text{N}}\text{}}$$is transmitted and then received at the channel receiver. ${\text{z}}_{\text{i}}\text{,}\text{i}\text{=1,2,}\cdots \text{,}\text{N}$are symbols denoting the ith quantization interval and not necessarily real numbers, hence, further decoding is required, as follows

$${\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{k}\text{)=}{\text{D}}_{\text{k}}\text{(}{\text{y}}_{\text{q}}\text{(}\text{k}\text{))}$$E8

where $${\text{D}}_{\text{k}}\text{(}\cdot \text{)}$$ is assumed to be a one to one mapping. A common decoding method (Curry, 1970) is$${\text{D}}_{\text{k}}\text{(}{\text{y}}_{\text{q}}\text{(}\text{k}\text{))=E{}\text{y}\text{(}\text{k}\text{)|}{\text{y}}_{\text{q}}\text{(}\text{k}\text{)}\text{=}{\text{z}}_l\text{},}\text{l}\text{=1,2,}\cdots \text{,}\text{N}$$

where E{.} is the operation of expectation.

3. Parameter identifiability

3.1. Definition of parameter identifiability

Reference (Zhang, 2003) proposed the definition of parameter identifiability for system (6) referring to the definition of state estimability under MMEE.

Let $\text{<mover accent="true">θ <mo>^</mo></mover>}\text{(}\text{k}\text{)}$be the MMEE estimation of $\text{θ}\text{(}\text{k}\text{)}$based on$\text{F}\text{(}\text{k}\text{)}$, and $\text{<mover accent="true">θ <mo>˜</mo></mover>}\text{(}\text{k}\text{)}\text{=}\text{θ}\text{(}\text{k}\text{)-}\text{<mover accent="true">θ <mo>^</mo></mover>}\text{(}\text{k}\text{)}$be the estimation error. Define the prior and posterior mean-square estimation error matrices respectively as$\text{Π}\text{(}\text{k}\text{)=E{(}\text{θ}\text{(}\text{k}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{))(}\text{θ}\text{(}\text{k}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{)}{\text{)}}^{\text{T}}\text{}}$$\text{P}\text{(}\text{k}\text{)=E{}\text{<mover accent="true">θ <mo>˜</mo></mover>}\text{(}\text{k}\text{)}{\text{<mover accent="true">θ <mo>˜</mo></mover>}}^{\text{T}}\text{(}\text{k}\text{)}}$

where$\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{)}$is the mean of$\text{θ}\text{(}\text{k}\text{)}$, i.e.$\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{)}\text{=}\text{E{}\text{θ}\text{(}\text{k}\text{)}}$.

Definition 1(Zhang, 2003): The linear system (1) with Gauss-Markov parameters (i.e. system (6)) is identifiable, if and only if

$$\text{I}\text{(}\text{θ}\text{(}\text{k}\text{);}\text{<mover accent="true">θ <mo>^</mo></mover>}\text{(}\text{k}\text{))>0,}\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E9

where $\text{I}\text{(}\cdot \text{;}\cdot \text{)}$denotes mutual information.

Based on Definition 1, the following conclusion was obtained in (Zhang, 2003).

Lemma 1(Zhang, 2003): The linear system (1) with Gauss-Markov parameters (i.e. system (6)) is identifiable, if and only if, the identifiability Gramian

${\text{W}}_{\text{k}}^{\text{id}}\text{=}{\displaystyle \sum _{\text{j}\text{=}\text{k}}^{\text{0}}{\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}}\text{Π}\text{(}\text{j}\text{)(}{\text{A}}^{\text{k}\text{-}\text{j}}{\text{)}}^{\text{T}},$

$$\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E10

has full rank, i.e.$\text{rank}\text{(}{\text{W}}_{\text{k}}^{\text{id}}\text{)=}\text{n}\text{+}\text{m}$,$\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$.

In the present paper, we propose an alternative definition of parameter identifiability for the quantized system (6)(7)(8) from information theoretic point of view, as follows.

Definition 2: The quantized linear system with Gauss-Markov parameters (6)(7)(8) is identifiable, if and only if

$$\text{I}\text{(}\text{θ}\text{(}\text{k}\text{);}{\text{Y}}_{\text{q}}^{\text{*}\text{k}}\text{)>0,}\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E11

where

$${\text{Y}}_{\text{q}}^{\text{*}\text{k}}\text{=[}{\text{y}}_{\text{q}}^{\text{*}}\text{(0) }{\text{y}}_{\text{q}}^{\text{*}}\text{(1) }\cdots {\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{k}\text{)}{\text{]}}^{\text{T}}.$$E12

Remark 1:

1. From information theory (Cover & Thomas, 2006), condition (11) is equivalent to that

$$\text{H}\text{(}\text{θ}\text{(}\text{k}\text{))>}\text{H}\text{(}\text{<mover accent="true">θ <mo>^</mo></mover>}\text{(}\text{k}\text{)),}\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E13

where $\text{H}\text{(}\cdot \text{)}$ denotes entropy, i.e. the prior error entropy $\text{H}\text{(}\text{θ}\text{(}\text{k}\text{))}$ is strictly greater than posterior error entropy$\text{H}\text{(}\text{<mover accent="true">θ <mo>^</mo></mover>}\text{(}\text{k}\text{))}$;

1. Definition 1 considers the mutual information between the actual value and estimate of parameters, so it relies on the estimation principle, while Definition 2 takes into account the intrinsic property of the system independent of the estimator used. If Definition 2 is adopted to analyze unquantized system (6), the identifiability condition (11) turns into

$$\text{I}\text{(}\text{θ}\text{(}\text{k}\text{);}{\text{Y}}_{}^{\text{k}}\text{)>0,}\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E14

where$${\text{Y}}^{\text{k}}\text{=[}\text{y}\text{(0) }\text{y}\text{(1) }\cdots \text{y}\text{(}\text{k}\text{)}{\text{]}}^{\text{T}}$$, and condition (9) is equivalent to (13) for linear Gaussian system (6) (Zhang, 2003). Hence, in some sense, Definition 2 is a more general one than Definition 1.

3.2. Identifiability analysis of quantized linear systems

Mutual information$\text{I}\text{(}\cdot \text{;}\cdot \text{)}$ (Cover & Thomas, 2006) is a measure of the information amount commonly contained in, and the statistic dependence between two random variables. $\text{I}\text{(}\cdot \text{;}\cdot \text{)}\ge \text{0}$with equality if and only if these two random variables are independent. Therefore (11) based on the information theoretic Definition 2 indicates that system is parameter identifiable if and only if any direction of parameter space is not orthogonal to all the past (quantized) measurements (Baram & Kailath, 1988), i.e. $$\forall \text{g}\in R^{\text{n}\text{+}\text{m}}\text{, }\text{g}\ne \text{0,}$$

$${\text{g}}^{\text{T}}\text{E{(}\text{θ}\text{(}\text{k}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{))(}{\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)-}{\text{<mover accent="true">y <mo>¯</mo></mover>}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{))}}\ne \text{0, }\exists \text{j}\le \text{k}\text{, }\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E15

where $${\text{<mover accent="true">y <mo>¯</mo></mover>}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}$$ is the mean of$${\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}$$, i.e.${\text{<mover accent="true">y <mo>¯</mo></mover>}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}=\text{E}\left\{{\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}\right\}$.

From the Gauss-Markov property of the parameters, we have

$$\text{θ}\text{(}\text{k}\text{)=}{\text{A}}^{\text{k}\text{-}j}\text{θ}\text{(}\text{j}\text{)+}{\displaystyle \sum _{\text{i}\text{=0}}^{\text{k}\text{-}\text{j}\text{-1}}{\text{A}}^{\text{k}\text{-}\text{j}\text{-}\text{i}\text{-1}}\text{Bw}\text{(}\text{j}\text{+}\text{i}\text{)}}$$E16

$$\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{)=}{\text{A}}^{\text{k}\text{-}\text{j}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}$$E17

where $${\displaystyle \sum _{\text{i}\text{=0}}^{\text{-1}}{\text{A}}^{\text{-}\text{i}\text{-1}}\text{Bw}\text{(}\text{k}\text{+}\text{i}\text{)}}\text{=0}$$ when $\text{j}\text{=}\text{k,}$ then

$$\begin{array}{l}\text{ E{(}\text{θ}\text{(}\text{k}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{))(}{\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)-}{\text{<mover accent="true">y <mo>¯</mo></mover>}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{))}}\\ \text{=E{(}\text{θ}\text{(}\text{k}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{))}\cdot {\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}}\\ \text{=E{[}{\text{A}}^{\text{k}\text{-}\text{j}}\text{(}\text{θ}\text{(}\text{j}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{))+}{\displaystyle \sum _{\text{i}\text{=0}}^{\text{k}\text{-}\text{j}\text{-1}}{\text{A}}^{\text{k}\text{-}\text{j}\text{-}\text{i}\text{-1}}\text{Bw}\text{(}\text{j}\text{+}\text{i}\text{)}}\text{]}\cdot {\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}}\\ \text{=}{\text{A}}^{\text{k}\text{-}\text{j}}\text{E{(}\text{θ}\text{(}\text{j}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{))}\cdot {\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}}\\ \text{=}{\text{A}}^{\text{k}\text{-}\text{j}}\text{E{}\text{θ}\text{'(}\text{j}\text{)}\cdot {\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}}\end{array}$$E18

where$$\text{θ}\text{'(}\text{j}\text{)=}\text{θ}\text{(}\text{j}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}$$. Let$$\text{θ}\text{'}\triangleq \text{θ}\text{'(}\text{j}\text{)}$$, $${\text{y}}_{\text{q}}^{\text{*}}\triangleq {\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}$$, and the time variable j of other relevant time-variant variables (e.g. F(j),$$\text{Π}\text{(}\text{j}\text{)}$$) is omitted for notational simplicity, then

$$\begin{array}{l}\text{E{}\text{θ}\text{'(}\text{j}\text{)}{\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)}}\triangleq \text{E{}\text{θ}\text{'}\cdot {\text{y}}_{\text{q}}^{\text{*}}\text{}=}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}{\displaystyle \underset{\text{θ}\text{'}\in R^{\text{n}\text{+}\text{m}}}{\int }\text{θ}\text{'}\cdot {\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)}\cdot \text{p}\text{(}\text{θ}\text{',}{\text{y}}_{\text{q}}^{\text{*}}\text{=}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{))}}}\text{d}\text{θ}\text{'}\\ \text{ =}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)}{\displaystyle \underset{\text{θ}\text{'}\in R^{\text{n}\text{+}\text{m}}}{\int }\text{θ}\text{'}\text{p}\text{(}\text{θ}\text{',}{\text{y}}_{\text{q}}^{\text{*}}\text{=}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{))}}}\text{d}\text{θ}\text{'}\end{array}$$E19

Based on the Bayesian law,

$$\begin{array}{l}\text{p}\text{(}\text{θ}\text{',}{\text{y}}_{\text{q}}^{\text{*}}\text{=}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{))}\\ \text{=}\text{p}\text{(}{\text{y}}_{\text{q}}^{\text{*}}\text{=}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)|}\text{θ}\text{')}\text{p}\text{(}\text{θ}\text{')}\\ \text{=}\text{p}\text{(}{\text{a}}_{\text{l}}\text{<}\text{y}\le {\text{a}}_{\text{l}\text{+1}}\text{|}\text{θ}\text{')}\text{p}\text{(}\text{θ}\text{')}\\ \text{=}\text{p}\text{(}{\text{a}}_{\text{l}}\text{<}{\text{F}}^{\text{T}}\text{θ}\text{+}\text{e}\le {\text{a}}_{\text{l}\text{+1}}\text{|}\text{θ}\text{')p(}\text{θ}\text{')}\\ \text{=}\text{p}\text{(}{\text{a}}_{\text{l}}\text{<}{\text{F}}^{\text{T}}\text{(}\text{θ}\text{'+}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{)+}\text{e}\le {\text{a}}_{\text{l}\text{+1}}\text{|}\text{θ}\text{')}\text{p}\text{(}\text{θ}\text{')}\\ \text{=}\text{p}\text{(}{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{'<}\text{e}\le {\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{'|}\text{θ}\text{')}\text{p}\text{(}\text{θ}\text{')}\\ \text{=}\text{p}\text{(}{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{'<}\text{e}\le {\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{')}\text{p}\text{(}\text{θ}\text{')}\end{array}$$E20

where the last equality is based on the fact that$$\text{θ}\text{'(}\text{j}\text{)}$$ and e(j) are stochastically independent, then

$$\begin{array}{l}{\displaystyle \underset{\text{θ}\text{'}\in R^{\text{n}\text{+}\text{m}}}{\int }\text{θ}\text{'}\text{p}\text{(}\text{θ}\text{',}{\text{y}}_{\text{q}}^{\text{*}}\text{=}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{))}}\text{d}\text{θ}\text{'}\\ \text{=}{\displaystyle \underset{\text{θ}\text{'}\in R^{\text{n}\text{+}\text{m}}}{\int }\text{θ}\text{'}\text{p}\text{(}{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{'<}\text{e}\le {\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{')}\text{p}\text{(}\text{θ}\text{')}}\text{d}\text{θ}\text{'}\\ \text{=}{\displaystyle \underset{\text{θ}\text{'}\in R^{\text{n}\text{+}\text{m}}}{\int }\text{θ}\text{'(}\text{T}\text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{'}}{\sqrt{\text{R}}}\text{)-}\text{T}\text{(}\frac{{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{-}{\text{F}}^{\text{T}}\text{θ}\text{'}}{\sqrt{\text{R}}}\text{))}\text{G}\text{(}\text{θ}\text{',0,}\text{Π}\text{)}}\text{d}\text{θ}\text{'}\end{array}$$E21

where T(.) is the probability distribution function of standardized normalized distribution (note that, T(.) is different from the tail function defined in (Ribeiro et al., 2006; You et al., 2011)), $$\text{G}\text{(}\text{θ}\text{',0,}\text{Π}\text{)}$$denotes the probability density function which means that stochastic vector $$\text{θ}\text{'}$$is Gaussian with zero-mean and covariance$$\text{Π}$$. Following the similar line of argument as in (Ribeiro et al., 2006; You et al., 2011), we calculate the integral (20) and then obtain

$${\displaystyle \underset{\text{θ}\text{'}\in R^{\text{n}\text{+}\text{m}}}{\int }\text{θ}\text{'}\text{p}\text{(}\text{θ}\text{',}{\text{y}}_{\text{q}}^{\text{*}}\text{=}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{))}}\text{d}\text{θ}\text{'=(}\varphi \text{(}\frac{{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}}{\sqrt{{\text{F}}^{\text{T}}\text{ΠF}\text{+}\text{R}}}\text{)-}\varphi \text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{<mover accent="true">θ <mo>¯</mo></mover>}}{\sqrt{{\text{F}}^{\text{T}}\text{ΠF}\text{+}\text{R}}}\text{))}\frac{\text{ΠF}}{\sqrt{{\text{F}}^{\text{T}}\text{ΠF}\text{+}\text{R}}}$$E22

where $$\varphi \text{(}\cdot \text{)}$$ is the probability density function of standardized normalized distribution. Substitute (21) into (18), we have

$$\text{E{}\text{θ}\text{'(}\text{j}\text{)}{\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)=}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}\frac{{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}}\text{(}\varphi \text{(}\frac{{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{) -}\varphi \text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{))}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)}\text{.}$$E23

Combining (22) with (17) and (14), we get that $${\text{g}}^{\text{T}}\text{E{(}\text{θ}\text{(}\text{k}\text{)-}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{k}\text{))(}{\text{y}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{)-}{\text{<mover accent="true">y <mo>¯</mo></mover>}}_{\text{q}}^{\text{*}}\text{(}\text{j}\text{))}}\ne \text{0}$$ is equivalent to

$$\begin{array}{l}{\text{g}}^{\text{T}}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}\frac{{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}}\text{(}\varphi \text{(}\frac{{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{) - }\varphi \text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{))}{\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)}\ne \text{0, }\\ \exists \text{j}\le \text{k}\text{, }\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1 }\text{.}\end{array}$$E24

Let

$$\text{ψ}\text{(}\text{j}\text{)}\triangleq {\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}\frac{{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j)Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}}\text{(}\varphi \text{(}\frac{{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j)Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{)-}\varphi \text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j)Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{))}$$E25

Then (23) is equivalent to that

$${\text{W}}_{\text{k}}^{\text{idq}}\text{=}{\displaystyle \sum _{\text{j}\text{=}\text{k}}^{\text{0}}{\text{ψ}}_{}^{\text{2}}\text{(}\text{j}\text{)}{\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)(}{\text{A}}^{\text{k}\text{-}\text{j}}{\text{)}}^{\text{T}}}\text{, }\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E26

has full rank. We conclude the above analysis as follows.

Theorem 1: The quantized linear system with Gauss-Markov parameters (6)(7)(8) is parameter identifiable, if and only if

$$\text{rank}{\text{W}}_{\text{k}}^{\text{idq}}\text{=}\text{n}\text{+}\text{m}\text{, }\forall \text{k}\ge \text{n}\text{+}\text{m}\text{-1}$$E27

Remark 2:

1. In Theorem 1, $$\text{ψ}\text{(}\text{j}\text{),}\text{j}\text{=0,1,2,}\cdots \text{,}\text{k}$$is defined by the quantizer, while$${\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)(}{\text{A}}^{\text{k}\text{-}\text{j}}{\text{)}}^{\text{T}}$$,$$\text{j}\text{=0,1,2,}\cdots \text{,}\text{k}$$ , which is the same part as in the unquantized system, reflects the intrinsic properties of the system. Hence, the full rank requirement of ${\text{W}}_{\text{k}}^{\text{idq}}$shows that the parameter identifiability of the quantized system is defined by quantizer and intrinsic properties of the system jointly;

2. When quantization level$$\text{N}\text{=1}$$, i.e.$${\text{a}}_{\text{1}}\text{=-}\infty$$, $${\text{a}}_2\text{=+}\infty$$, $$\text{ψ}\text{(}\text{j}\text{)}\equiv \text{0}$$, then${\text{W}}_{\text{k}}^{\text{idq}}\equiv \text{0}$, condition (26) is not satisfied and the system is not identifiable. This is consistent with the intuition. From (10) and (25), it can be observed that the difference between unquantized estimability Gramian ${\text{W}}_{\text{k}}^{\text{id}}$ and quantized estimability Gramian ${\text{W}}_{\text{k}}^{\text{idq}}$ is that the later includes additional weights$${\text{ψ}}^{\text{2}}\text{(}\text{j}\text{),}\text{j}\text{=0,1,2,}\cdots \text{,}\text{k}$$. As a result, it can be seen that besides the situation of quantization level$$\text{N}\text{=1}$$, the matrix ${\text{W}}_{\text{k}}^{\text{idq}}$ may become singular due to the property of$${\text{ψ}}^{\text{2}}\text{(}\text{j}\text{)}$$, though ${\text{W}}_{\text{k}}^{\text{id}}$ has full rank;

3. The quantizer in Theorem 1 is time-invariant. However, by using the above analysis method, a conclusion similar to Theorem 1 can be derived for time-variant quantizer, except that the weights in the estimability Gramian reflects the time-variant property of the quantizer, i.e.,

$$\text{ψ}\text{(}\text{j}\text{)}\triangleq {\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}\frac{{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{(}\text{j}\text{))}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j)Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}}\text{(}\varphi \text{(}\frac{{\text{a}}_{\text{l}}\text{(}\text{j}\text{)-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j)Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{)-}\varphi \text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{(}\text{j}\text{)-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j)Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{))}$$E28

1. Condition (26) is equivalent to that the matrix sequence $$\text{{}\text{ψ}\text{(}\text{j}\text{)}{\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{),}\text{j}\le \text{k}\text{}}$$ has column rank n+m. $$\text{{}\text{ψ}\text{(}\text{j}\text{)}{\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{),}\text{j}\le \text{k}\text{}}$$can be decomposed as$$\text{{}\text{ψ}\text{(}\text{j}\text{)}{\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{),}\text{j}\le \text{k}\text{}}\text{diag}\text{{}\text{ψ}\text{(0),}\text{ψ}\text{(1),}\cdots \text{,}\text{ψ}\text{(}\text{k}\text{)}}$$, where$$\text{diag}\text{{}\cdot \text{}}$$denotes diagonal matrix. Hence the parameter identifiability of the original system can be preserved if$$\text{ψ}\text{(}\text{j}\text{)}\ne \text{0,}\text{j}\text{=0,1,2,}\cdots$$. Especially, condition (26) is equivalent to Lemma 1 for (6). Suppose we can design a time-variant quantizer as follows

$$\begin{array}{l}{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{(}\text{j}\text{))=}{\text{c}}_{\text{l}}\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}\text{,}\text{l}\text{=1,2,}\cdots \text{,}\text{N}\text{,}\\ {\text{a}}_{\text{l}}\text{(}\text{j}\text{)=}{\text{d}}_{\text{l}}\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}\text{+}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{), }\text{l}\text{=1,2,}\cdots \text{,}\text{N}\text{+1 , }\text{j}\text{=0,1,2,}\cdots \text{,}\text{k}\text{ ,}\end{array}$$E29

where $${\text{c}}_{\text{l}}$$ and $${\text{d}}_{\text{l}}$$ are constants which make

$$\text{ψ}\text{(}\text{j}\text{)=}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}{\text{c}}_{\text{l}}\text{(}\varphi \text{(}{\text{d}}_{\text{l}}\text{)-}\varphi \text{(}{\text{d}}_{\text{l}\text{+1}}\text{))}}\ne \text{0}$$E30

be the same for every j, thus,

$${\text{W}}_{\text{k}}^{\text{idq}}\text{=(}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}{\text{c}}_{\text{l}}\text{(}\varphi \text{(}{\text{d}}_{\text{l}}\text{)-}\varphi \text{(}{\text{d}}_{\text{l}\text{+1}}\text{))}}{\text{)}}^{\text{2}}{\displaystyle \sum _{\text{j}\text{=}\text{k}}^{\text{0}}{\text{ψ}}_{}^{\text{2}}\text{(}\text{j}\text{)}{\text{A}}^{\text{k}\text{-}\text{j}}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)(}{\text{A}}^{\text{k}\text{-}\text{j}}{\text{)}}^{\text{T}}}.$$E31

By comparing (30) with (10), we can find that such a time-variant quantizer does not change the parameter identifiability of the original system if and only if (29) is satisfied;

1. The parameter identifiability of the system can be preserved even if the quantization level is low as long as the quantizer is designed reasonably. Especially, when quantization level$$\text{N}\text{=2}$$, set$${\text{c}}_{\text{1}}\text{=-1}$$, $${\text{c}}_2\text{=1}$$, $${\text{d}}_{\text{1}}\text{=-}\infty$$, $${\text{d}}_2\text{=0}$$, $${\text{d}}_3\text{=+}\infty$$in the formula (28), then$$\text{ψ}\text{(}\text{j}\text{)}\equiv \sqrt{\text{2/}\text{π}}\text{,}\text{j}\text{=0,1,2,}\cdots$$, namely a coarse quantizer of 1 bit can preserve the parameter identifiability of the original system.

4. Convergence analysis

In this section, we discuss the convergence property of the Gramian in Theorem 1, i.e., the convergence property of$$\text{ψ}\text{(}\text{j}\text{),}\text{j}\text{=0,1,2,}\cdots$$.

We know that

$${\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}\frac{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}}\varphi \text{(}\frac{{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{)-}\varphi \text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{))=0}$$E32

(31)

by the property of$$\varphi \text{(}\cdot \text{)}$$, then $$\text{ψ}\text{(}\text{j}\text{)}$$ can be re-expressed as

$$\text{ψ}\text{(}\text{j}\text{)=}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}\frac{{\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}}\varphi \text{(}\frac{{\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{)-}\varphi \text{(}\frac{{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}}{\sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}}\text{))}$$E33

(32)

Let$$\text{Δ=}\underset{\text{1}\le \text{k}\le \text{N}}{\text{sup}}{\text{Δ}}_{\text{l}}$$,where$${\text{Δ}}_{\text{l}}\text{=}{\text{a}}_{\text{l}\text{+1}}\text{-}{\text{a}}_{\text{l}}$$, $$\text{l}\text{=1,2,}\cdots \text{,}\text{N}$$, let$${\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)}\triangleq {\text{a}}_{\text{l}}\text{-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}$$, then$${\text{Δ}}_{\text{l}}\text{=}{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}\text{+1}}\text{(}\text{j}\text{)-}{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)}$$, $$\sqrt{\cdot }\triangleq \sqrt{{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{Π}\text{(}\text{j}\text{)}\text{F}\text{(}\text{j}\text{)+}\text{R}}$$,$${\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}\triangleq {\text{D}}_{\text{j}}\text{(}{\text{z}}_{\text{l}}\text{)-}{\text{F}}^{\text{T}}\text{(}\text{j}\text{)}\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{(}\text{j}\text{)}$$. Consider the convergence property of the Gramian ${\text{W}}_{\text{k}}^{\text{idq}}$ when$$\text{Δ}\to \text{0}$$. Note that $$\text{N}\to \infty$$ when$$\text{Δ}\to \text{0}$$, and then

$$\begin{array}{l}\underset{\text{N}\to \infty }{\text{lim}}\text{ψ}\text{(}\text{j}\text{)=}\underset{\text{N}\to \infty }{\text{lim}}{\displaystyle \sum _{\text{l}\text{=1}}^{\text{N}}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}}\text{(}\varphi \text{(}\frac{{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{)-}\varphi \text{(}\frac{{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}\text{+1}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{))}\\ \text{ =}{\displaystyle \sum _{\text{l}\text{=1}}^{\infty }\underset{{\text{Δ}}_{\text{l}}\to \text{0}}{\text{lim}}}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{(}\varphi \text{(}\frac{{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{)-}\varphi \text{(}\frac{{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)+}{\text{Δ}}_{\text{l}}}{\sqrt{\cdot }}\text{))}\\ \text{ =}{\displaystyle \sum _{\text{l}\text{=1}}^{\infty }\underset{{\text{Δ}}_{\text{l}}\to \text{0}}{\text{lim}}}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\frac{\varphi \text{(}\frac{{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{)-}\varphi \text{(}\frac{{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)+}{\text{Δ}}_{\text{l}}}{\sqrt{\cdot }}\text{)}}{\text{-}\frac{{\text{Δ}}_{\text{l}}}{\sqrt{\cdot }}}\text{.(-}\frac{{\text{Δ}}_{\text{l}}}{\sqrt{\cdot }}\text{)}\\ \text{ =}{\displaystyle \sum _{\text{l}\text{=1}}^{\infty }\underset{{\text{Δ}}_{\text{l}}\to \text{0}}{\text{lim}}}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\frac{\text{d}}{\text{d}\text{s}}\varphi \text{(}\text{s}\text{)}{\text{|}}_{\text{s}\text{=}\frac{{\text{<mover accent="true">a <mo>˜</mo></mover>}}_{\text{l}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}}\text{(-}\frac{{\text{Δ}}_{\text{l}}}{\sqrt{\cdot }}\text{)}\\ \text{ =}{\displaystyle \sum _{\text{l}\text{=1}}^{\infty }\underset{{\text{Δ}}_{\text{l}}\to \text{0}}{\text{lim}}}{\text{(}}^{\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}}\frac{\text{exp{-}\frac{\text{1}}{\text{2}}{\text{(}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{)}}^{\text{2}}\text{}}}{\sqrt{\text{2}\text{π}}}\frac{{\text{Δ}}_{\text{l}}}{\sqrt{\cdot }}\\ \text{ =}{\displaystyle \underset{R}{\int }{\text{(}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{)}}^{\text{2}}\frac{\text{exp{-}\frac{\text{1}}{\text{2}}{\text{(}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\text{)}}^{\text{2}}\text{}}}{\sqrt{\text{2}\text{π}}}}\frac{\text{d}{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }}\end{array}$$E34

Let $$\text{r}\text{=}\frac{{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }},$$ then $$\text{d}\text{r}\text{=}\frac{\text{d}{\text{z}}_{\text{l}}^{\text{*}}\text{(}\text{j}\text{)}}{\sqrt{\cdot }},$$

$$\underset{\text{N}\to \infty }{\text{lim}}\text{ψ}\text{(}\text{j}\text{)=}{\displaystyle \underset{R}{\int }{\text{r}}^{\text{2}}\frac{{\text{e}}^{\text{-}\frac{{\text{r}}^{\text{2}}}{\text{2}}}}{\sqrt{\text{2}\text{π}}}}\text{d}\text{r}\text{=1}\text{.}$$E35

Remark 3: Equation (34) implies the convergence of Theorem 1 to Lemma 1 when$$\text{Δ}\to \text{0}$$, i.e. the quantized identifiability Gramian ${\text{W}}_{\text{k}}^{\text{idq}}$ converges to the unquantized identifiability Gramian${\text{W}}_{\text{k}}^{\text{id}}$.

5. Simulation

In order to illustrate our main conclusion, the following system is simulated with the tool of Matlab:$\text{y}\text{(}\text{k}\text{)=}\text{b}\text{(}\text{k}\text{)}\text{u}\text{(}\text{k}\text{-1)+}\text{e}\text{(}\text{k}\text{)}$

where $\text{b}\text{(}\text{k}\text{)}$ is the parameter to be identified, and can be modeled as a Gauss-Markov process. Then the system model can be transformed to$\{\begin{array}{l}\text{θ}\text{(}\text{k}\text{+1)=}\text{aθ}\text{(}\text{k}\text{)+}\text{w}\text{(}\text{k}\text{)}\\ \text{y}\text{(}\text{k}\text{)=}{\text{F}}^{\text{T}}\text{(}\text{k}\text{)}\text{θ}\text{(}\text{k}\text{)+}\text{e}\text{(}\text{k}\text{)}\end{array}$

where$\text{θ}\text{(}\text{k}\text{)=}\text{b}\text{(}\text{k}\text{)}$, a=0.5.$$\text{e}\text{(}\text{k}\text{)}$$, $$\text{w}\text{(}\text{k}\text{)}$$and $\text{θ}\text{(0)}$ are mutually statistically uncorrelated, their covariance are Q=1, R=0.1, $\text{Π}\text{(0)=1}$respectively, and the mean of $\text{θ}\text{(0)}$ is$\text{<mover accent="true">θ <mo>¯</mo></mover>}\text{=1}$. Here we set $\text{F}\text{(}\text{k}\text{)=}\text{u}\text{(}\text{k}\text{-1)=2sin(}\text{k}\text{)+3}$ as the assumed system input (i.e., the control signal, which can be considered to be generated, for example, by the adaptive controller), where the additive term “3” plays the role of avoiding the problem of “turn-off” (Astrom & Wittenmark, 1994).

To do the illustrative simulation, an optimal filter is required though the analysis about parameter identifiability is independent of the estimator used. The discussed linear system with Gauss-Markov parameter is transformed to state space model, and then the problem of parameter identification can be treated as states estimation. A number of quantized state estimators have been proposed by scholars in various areas, and we choose the Gaussian fit algorithm (Curry, 1970) as the filter in this section for that this filter which bases on the Gaussian assumption is near optimal and convenient to be implemented. Note that, in this simulated model, $\text{F}\text{(}\text{k}\text{)}$is defined by $\text{u}\text{(}\text{k}\text{-1)}$ completely, so it is known at the channel receiver; however, in general model (1) ${\text{a}}_{\text{i}}\text{(}\text{k}\text{),}\text{i}\text{=1,2,}\cdots \text{,}\text{n}$and ${\text{b}}_{\text{i}}\text{(}\text{k}\text{),}\text{i}\text{=1,2,}\cdots \text{,}\text{m}$ are to be identified, then $\text{F}\text{(}\text{k}\text{)}$ is defined by $$\text{u}\text{(}\text{k}\text{-1)}\cdots \text{u}\text{(}\text{k}\text{-}\text{m}\text{)}$$ and $$\text{-}\text{y}\text{(}\text{k}\text{-1)}\cdots \text{-}\text{y}\text{(}\text{k}\text{-}\text{n}\text{)}$$ jointly. So in general, the quantized signals$${\text{y}}_{\text{q}}\text{(}\text{k}\text{-1),}\cdots \text{,}{\text{y}}_{\text{q}}\text{(}\text{k}\text{-}\text{n}\text{)}$$, instead of the actual outputs $$\text{y}\text{(}\text{k}\text{-1),}\cdots \text{,}\text{y}\text{(}\text{k}\text{-}\text{n}\text{)}$$ are received at the channel receiver, thus $$\text{y}\text{(}\text{k}\text{-1),}\cdots \text{,}\text{y}\text{(}\text{k}\text{-}\text{n}\text{)}$$ in $\text{F}\text{(}\text{k}\text{)}$ should be replaced by their estimates.

The analysis about parameter identifiability of quantized systems is suitable for any rational quantizer. Here the Max-Lloyd quantizer (Proakis, 2001) generally adopted in areas of communication and signal processing is employed. In the following statement, cases of quantization level $$\text{N}\text{=4}$$ and $$\text{N}\text{=2}$$ in (7) are simulated, respectively. The thresholds of the 4 level Max-Lloyd quantizer are {–∞, –0.9816, 0, 0.9816, +∞} and the outputs of the quantizer are {–1.51, –0.4528, 0.4528, 1.51} when the signal to be quantized is standardized normally distributed. In the case of 2 level quantizer, the thresholds are {–∞, 0, +∞} and the outputs of the quantizer are {–0.7979, 0.7979}. Hence the thresholds of the time-variant quantizers with 4 and 2 levels are respectively $${\text{σ}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${–∞, –0.9816, 0, 0.9816, +∞}+$${\text{E}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${1, 1, 1, 1, 1} and $${\text{σ}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${–∞, 0, +∞}+$${\text{E}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${1, 1, 1}, the outputs of the quantizers are $${\text{σ}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${–1.51, –0.4528, 0.4528, 1.51}+$${\text{E}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${1, 1, 1, 1} and $${\text{σ}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${–0.7979, 0.7979}+$${\text{E}}_{\text{y}\text{(}\text{k}\text{)}}\text{×}$${1, 1}, where $${\text{E}}_{\text{y}\text{(}\text{k}\text{)}}$$ and $${\text{σ}}_{\text{y}\text{(}\text{k}\text{)}}$$ are the mean and standard deviation of the output y(k) respectively.

It is obvious that the above model is parameter identifiable by Lemma 1 when it is unquantized. We get $$\text{ψ}\text{(}\text{j}\text{)}\equiv \text{0}\text{.8823}$$ by calculating the weight $$\text{ψ}\text{(}\text{j}\text{)}$$ in equation (27) when the 4 level time-variant Max-Lloyd quantizer is used and $$\text{ψ}\text{(}\text{j}\text{)}\equiv \text{0}\text{.6366}$$ when quantization level$$\text{N}\text{=2}$$. Hence the parameter identifiability will not be changed by the quantization according to Remark 2, i.e. the quantized system is still parameter identifiable, theoretically. The simulation results of the quantized system shown in Fig. 1 ($$\text{N}\text{=4}$$) and Fig. 2 ($$\text{N}\text{=2}$$) illustrate the above conclusion.

In Fig. 1(a) and Fig. 2(a), actual values of parameter are denoted by solid lines and the estimates are denoted by dotted lines. Estimation errors are shown in Fig. 1(b) and Fig. 2(b). Fig. 1 and Fig. 2 show that the estimate can track the real value of the parameter when the outputs are quantized coarsely. The curves of prior error entropy and posterior error entropy are shown in Fig. 1(c) and Fig. 2(c). The entropy is calculated by

$$\text{H}\text{(}\text{x}\text{)=}\frac{\text{n}}{\text{2}}\text{ln2}\text{πe}\text{+}\frac{\text{1}}{\text{2}}\text{ln}\left|\text{C}\right|$$E36

where $\text{x}\in R^{\text{n}}$ is a Gaussian vector with covariance C, $\left|\begin{array}{c}\cdot \end{array}\right|$denotes determinant. For quantized systems, the probability distribution of estimation error $\text{<mover accent="true">θ <mo>˜</mo></mover>}\text{(}\text{k}\text{)}$ is unknown, but is supposed to make the entropy of $\text{<mover accent="true">θ <mo>˜</mo></mover>}\text{(}\text{k}\text{)}$ maximal according to “maximal entropy principle” of Jaynes (Jaynes, 1957), namely, the uncertainty of $\text{<mover accent="true">θ <mo>˜</mo></mover>}\text{(}\text{k}\text{)}$ is supposed to be maximal in the situation of lack of prior information, hence $\text{<mover accent="true">θ <mo>˜</mo></mover>}\text{(}\text{k}\text{)}$ is assumed to be Gaussian, and thus (35) can be adopted to calculate the entropy of $\text{<mover accent="true">θ <mo>˜</mo></mover>}\text{(}\text{k}\text{)}$ in this simulation. We can observe from Fig. 1(c) and Fig. 2(c) that the posterior error entropy is strictly smaller than prior error entropy from the initial time instant. This indicates that this quantized system is parameter identifiable, and these observations consist with our analysis mentioned above perfectly. Besides, we can observe that the estimation error when quantization level $$\text{N}\text{=2}$$ is greater than that in the case of quantization level $$\text{N}\text{=4}$$ though the system is parameter identifiable in both of the two quantization cases. This shows that systems with different quantizers lead to different estimation precision, though all of them are parameter identifiable when rational quantizers are used.

Figure 1.

a) Actual value and estimate of b(k), N=4, (b) Estimation error of b(k), N=4, (c) Prior and posterior error entropy, N=4

Figure 2.

a) Actual state and estimate of b(k), N=2, (b) Estimation error of b(k), N=2, (c) Prior and posterior error entropy, N=2

6. Conclusion

This paper discusses the parameter identifiability of quantized linear systems with Gauss-Markov parameters from information theoretic point of view. The existing definition concerning this property is reviewed and new definition is proposed for quantized systems. Criterion function, the Gramian of parameter identifiability for quantized systems is analyzed based on the quantity of mutual information. The derived conclusions consist with our intuition very well and also provide us with intrinsic perspective for the quantizer design. The analysis shows that the Gramian of quantized systems converge to that of unquantized systems when the quantization intervals turn to zero, and a well designed quantizer can preserve the identifiability of the original system even if the quantizer is as coarse as one bit. The analytical analysis is verified by the illustrative simulation.