State-aware Real-time Tracking and Remote Reconstruction of a Markov Source

I. INTRODUCTION

TODAY’S communication networks are in a transitional phase to supporting cyber-physical and interactive critical systems, which are key enablers for a plethora of new services and applications, such as autonomous transportation, industrial robotics, telehealth, and environmental monitoring. Emerging real-time autonomous systems, empowered with networked agents with advanced processing and learning capabilities, are expected to take advantage of network and sensing data and transform both human and digital decision making. Nevertheless, the realization of this euphoric vision hinges upon networks’ ability to timely and effectively gather, analyze, and transport vast new sources of data. As a step in that direction, a radically new approach, which accounts for the semantics of information, defined as the importance and the goal-oriented utility of data exchanged in a network, has emerged. Reconsidering the entire communication process under the prism of semantics of information is instrumental in transforming the way we generate, transmit, and reconstruct data in timesensitive and data-intensive communication systems. Anthony Ephremides is among the very first who proposed and advocated for the concept of semantics of information, laying the foundation stones of goal-oriented semantic communications. A highly relevant yet challenging problem in this context is to design joint source sampling, transmission, and reconstruction techniques, which consider the dynamics of the information source and enable real-time remote tracking with the objective of actuation.

Most prior work on remote tracking to date has mainly focused on proposing sampling or scheduling policies aiming to minimize estimation error or mean square error, letting the significance and the usefulness of the generated and transmitted information with respect to the application-driven goal and context aside. In contrast to these works, in this paper, we propose a new state-aware sampling and transmission policy and introduced a new importance-aware error metric, unearthing the prominent role of having different action probabilities for different states.

A. Related Work

The problem of scheduling in event-triggered estimation has been considered in [1]–[6], where a sensor observes the state of a process and transmits it to the receiver only when certain events occur. Optimal sampling and transmission policies for noiseless communication channels are proposed in [7]–[9]. The study in [7] considers sequential estimation with limited information, where an observer sequentially observes a stochastic process and sends the resulting sample to a receiver over a noiseless communication channel. The authors in [8] study a remote estimation problem in a noiseless communication system in the presence of an energy harvesting sensor and a remote estimator. [9] presents an optimal threshold transmission policy for a noiseless communication system where a sensor observes a first-order Markov process and transmits the sample to the receiver.

The work [10] proposes an optimal transmission strategy in two sensor-assisted Gauss-Markov systems, extended to multiple sensors and processes in [11]. The work [12] analyzes the optimal estimation and transmission policies for remote estimation over time-varying packet drop channels. This study involves a scenario representing the information source as a finite-state Markov chains and first-order auto-regressive processes. The authors in [13] and [14] study the fundamental limits and trade-offs of remote estimation of Markov processes under communication constraints.

Optimal sampling and remote estimation for monitoring real-time stochastic processes is studied in [15]–[18]. The problem of estimating the current state of a dynamic process using previous measurements and a linear time-invariant discrete-time (LTI) model of the process is investigated in [19]–[21]. The main objective of the aforementioned studies is to present sampling and transmission strategies that minimize estimation errors, disregarding the importance of information with respect to its utilization. Metrics that capture the semantics and effectiveness of information, leveraging synergies between data processing, information transmission, and signal reconstruction have recently been introduced in [22]–[35].

B. Contributions

In this work, we consider the problem of real-time remote tracking of an information source in a time slotted communication system. A sampler performs sampling of a two-state Markov process, and then the transmitter sends the sample in the form of packets to a remote receiver over an unreliable wireless channel. Then, the real-time reconstruction of the information source is performed at the receiver based on the successfully received samples. The system is considered to be in a synced state if the source state matches the state of the reconstructed source, otherwise the system is in an erroneous state. Furthermore, the receiver performs a specific action according to the estimated state of the information source. This paper extends the results of [23], [34] in which the problem of real-time tracking and reconstruction of an information source with the purpose of actuation is studied. These papers proposed semantics-empowered policies to achieve significant reduction in both the real-time reconstruction and the cost of actuation errors. In this work, we introduce a new stateaware sampling and transmission policy, and we evaluate its performance in terms of a set of semantics-aware metrics that capture the significance of information and various characteristics of the system’s performance. Our key contributions are summarized as follows:

1) We propose a state-aware randomized stationary sampling and transmission policy, in which we consider different sampling and success probabilities for different states of the information source. This becomes relevant to scenarios where the states encode commands for actuation or other potential tasks, where different actions have different importance, thus, is important to allow for different sampling frequencies.

2) We analyze the performance of the proposed strategy in terms of time-averaged reconstruction error, cost of actuation error, and consecutive error metrics, and we compare it with previous proposed joint sampling and transmission policies [23], [33], [34].

3) We define a new timing-aware error metric, namely importance-aware consecutive error metric, which jointly captures both timing- and importance-related aspects of errors. Specifically, this metric measures the impact on the performance when the system remains in a specific erroneous state for several consecutive time slots.

4) We solve the optimization problem of minimizing the average cost of actuation error subject to a time-averaged sampling cost constraint, as a means to reveal when and under which conditions the proposed state-aware randomized stationary policy outperforms state-of-the-art alternatives. [TeX:] $$\{X(t), t \in \mathbb{N}\},$$ depicted in Fig. 2. Therein, the self-transition probability and the probability of transition to another state at time slot t + 1 are defined as follows

II. SYSTEM MODEL

We consider a time slotted communication system in which a sampler performs sampling of an information source X(t) at time slot t, after which the transmitter sends the sample to the receiver over a wireless channel, as shown in Fig. 1. The remote receiver operates as an actuator and performs actions based on the reconstructed state of the information source. We model the information source as a two-state discrete time Markov chain (DTMC)

In this paper, we consider different sampling and transmission actions for the states of the information source. We denote the action of sampling at time slot t when the information source is at state i (i = 0, 1) by [TeX:] $$\alpha_i^{\mathrm{S}}(t),$$ where [TeX:] $$\alpha_i^{\mathrm{S}}(t)=1$$ if the source at state i is sampled and [TeX:] $$\alpha_i^{\mathrm{s}}(t)=0$$ otherwise. Furthermore, when [TeX:] $$\alpha_i^{\mathrm{s}}(t)=1,$$ the action of transmitting the sample is denoted by [TeX:] $$\alpha_i^{\mathrm{tx}}(t),$$ where [TeX:] $$\alpha_i^{\mathrm{tx}}(t)=1$$ if the sample is transmitted, otherwise the transmitter remains idle, [TeX:] $$\alpha_i^{\mathrm{tx}}(t)=0.$$ At time slot t, the receiver constructs an estimate of the process X(t), denoted by [TeX:] $$\hat{X}(t)$$ based on successfully received samples. The channel state [TeX:] $$h_i(t)$$ is equal to 1 if the information source at state i is sampled and successfully decoded by the receiver, and 0 otherwise. We define the success probability when the information source at state i is sampled and transmitted, as [TeX:] $$p_{\mathrm{s}_i}=\operatorname{Pr}\left[h_i(t)=1\right] .$$ Note that allowing for different success probabilities can have interesting connections with performing simple state-aware power control. Successful/failed transmissions are declared to the transmitter using acknowledgment (ACK)/negative-ACK packets, which are assumed to be delivered instantaneously and error free to the transmitter¹. Therefore, the transmitter has perfect knowledge of the reconstructed source state at time slot t, i.e., [TeX:] $$\hat{X}(t)$$². We also assume that a sample is discarded when its transmission fails.

¹ Actually, only the semantics-aware policy requires an ACK/NACK feedback channel.

² In this paper, since we do not consider any decoder/estimation policy, the current state at the transmitter is the latest received update.

III. SAMPLING AND TRANSMISSION POLICIES

We propose a new sampling and transmission policy, coined state-aware randomized stationary, in which the generation of a sample is triggered in a probabilistic manner at each time slot. Specifically, we introduce a scheme that allows for assigning different sampling probabilities for different states, adjusting the sampling frequency depending on the importance of the state. Consider, for example, the scenario where each state is a command for a remote agent that requires to be executed, and different commands are of different importance or criticality. We assume that [TeX:] $$p_{\alpha_i^{\mathrm{s}}}$$ is the probability of joint sampling and transmission actions when the source is at the state i. Therefore, we define [TeX:] $$p_{\alpha_i^{\mathrm{s}}}$$ as follows

The probability that the source at the state i is not sampled at time slot t+1 is [TeX:] $$\operatorname{Pr}\left[\alpha_i^{\mathrm{s}}(t+1)=0\right]=1-p_{\alpha_i^{\mathrm{s}}} .$$ In addition, for comparison, we adopt three relevant policies proposed in [23], [34]. Below we provide a short description of them.

1) Uniform: Sampling is conducted periodically every d time slots, independently of the evolution of the source X(t). Therefore, the sampling time sequences are [TeX:] $$\left\{t_k=k d, k \geqslant 1\right\}.$$ While this policy is simple and easy to implement, several state transitions can be missed during the time interval between two performed samples.

2) Change-aware: A new sample is generated when a change in the state of the source X(t) is observed between two consecutive time slots without considering whether the system is in sync or not.

3) Semantics-aware: When the system is in a sync state, i.e., [TeX:] $$X(t)=\hat{X}(t)$$, a sample is generated if a change in the source state is observed at the next time slot, i.e., [TeX:] $$X(t+1) \neq X(t)$$. However, when the system is in an erroneous state, i.e., [TeX:] $$X(t) \neq \hat{X}(t)$$, a sample is generated if the source state at the next time slot is not equal to the state of the reconstructed source at time slot t, i.e., [TeX:] $$X(t+1) \neq \hat{X}(t)$$.

IV. PRELIMINARY PERFORMANCE ANALYSIS

In this section, we analyze the performance of state-aware randomized stationary policy in terms of time-averaged reconstruction error and average cost of actuation error.

A. Real-time Reconstruction Error

The real-time reconstruction error captures the discrepancy between the original source X(t) and the reconstructed source [TeX:] $$\hat{X}(t)$$, at time slot t, i.e.,

where at time slot t, E(t) = 0 denotes the system is in the sync state, while the erroneous state of the system is denoted by [TeX:] $$E(t) \neq 0$$. The time-averaged reconstruction error or the probability that the system is in an erroneous state, [TeX:] $$P_E,$$ for an observation interval [1, T] with T being a large positive number, is defined as [23], [34]

Where [TeX:] $$\mathbb{1}(\cdot)$$ is the indicator function.

For a two-state DTMC information source, [TeX:] $$P_E$$ in (4) is given by

Note that [TeX:] $$\pi_{0,1} \text { and } \pi_{1,0}$$ are the probabilities obtained from the stationary distribution of the two-dimensional DTMC describing the joint status of the system regarding the current state at the original source, i.e., [TeX:] $$(X(t), \hat{X}(t)).$$ To derive [TeX:] $$\pi_{i, j},(i, j) \in\{0,1\},$$ we assume that when the sampler performs sampling, the transmitter sends the sample in the form of packets during the same time slot.

Lemma 1. For a two-state DTMC information source, the stationary distribution [TeX:] $$\pi_{i, j}$$ for the state-aware randomized stationary policy is given by³

³ This work can be extended to more than two-state DTMC information source. In Appendix B, we provide an example for the three-state DTMC information source.

where

Proof. See Appendix A.

Using (6b) and (6c), the time-averaged reconstruction error in (4) can be calculated as

where [TeX:] $$\Phi\left(p_{\alpha_0^{\mathrm{s}}}, p_{\alpha_1^{\mathrm{s}}}\right)$$ is given in (7).

B. Cost of Actuation Error

This metric captures the significance of the error at the receiver side and considers different cost or penalties for different erroneous actions. To study the cost of actuation error we define [TeX:] $$C_{i, j}$$ the cost of error when the current state of the source is i, and the reconstructed source is in state [TeX:] $$j \neq i .$$ It is assumed that [TeX:] $$C_{i, j}$$ does not change over time. Now, using [TeX:] $$C_{i, j}$$, the average cost of actuation error for a two-state DTMC can be calculated as follows

where using (6b) and (6c), we can write (9) as

Remark 1. Using (39) and (41), we can prove that when [TeX:] $$\max \left\{0, T_1\right\} \leqslant p_{\alpha_1^s} \leqslant 1,$$ the state-aware randomized stationary policy has lower average cost of actuation error as compared to the semantics-aware policy for [TeX:] $$\max \left\{0, T_2\right\} \leqslant p_{\alpha_0^s} \leqslant 1,$$ where [TeX:] $$T_1 \text { and } T_2$$ are given by

Also, when [TeX:] $$0 \leqslant p_{\alpha_1^s} \leqslant \min \left\{0, T_1\right\}$$ and [TeX:] $$0 \leqslant p_{\alpha_0^s} \leqslant \min \left\{0, T_2\right\} \text {, }$$ the state-aware randomized stationary policy has lower average cost of actuation error in comparison with the semantics-aware policy.

Remark 2. We can analytically prove that for [TeX:] $$\frac{q^2 p_{\alpha_0^s}^s p_{s_0}}{p_{s_1}\left[1-p_{\alpha_0^s}^s p_{s_0}(1+q(1-q))\right]} \leqslant p_{\alpha_1^s} \leqslant 1,$$ the timeaveraged reconstruction error and the average cost osf actuation error are decreasing with p, when [TeX:] $$\sqrt{\frac{q p_{\alpha_0^s} p_{s_0}\left(q+(1-q) p_{\alpha_1^s} p_{s_1}\right)}{p_{\alpha_1^s} p_{s_1}\left(1-p_{\alpha_0^s} p_{s_0}\right)}}\lt p \leqslant 1.$$ Furthermore, when [TeX:] $$\frac{p^2 p_{\alpha_1^s} p_{s_1}}{p_{s_0}\left[1-p_{\alpha_1^s} p_{s_1}(1+p(1-p))\right]} \leqslant p_{\alpha_0^s} \leqslant 1 \text {, }$$ the time-averaged reconstruction error and the average csost of actuation error are decreasing with q, for [TeX:] $$\sqrt{\frac{p p_{\alpha_0^s} p_{s_0} p_{\alpha_1^s} p_{s_1}+p^2 p_{\alpha_1^s} p_{s_1}\left(1-p_{\alpha_0^s} p_{s_0}\right)}{p_{\alpha_0^s} p_{s_0}\left(1-p_{\alpha_1^s} p_{s_1}\right)}}\lt q \leqslant 1 .$$

V. JOINT TIMING AND IMPORTANCE ERROR METRICS

In this section we consider the impact of timing and importance of errors, and we propose an extension of the consecutive error metric, termed importance-aware consecutive errors, which takes into account jointly both the timing and the importance aspects of errors.

A. Consecutive Error Metric

The consecutive error metric, first introduced in [34], quantifies the number of consecutive time slots during which the system is in an erroneous state ⁴. This metric can be described by a DTMC as depicted in Fig. 3. At time slot t, [TeX:] $$C_E(t)=0$$ denotes the synced state, whereas [TeX:] $$C_E(t)=1 \leqslant i \leqslant n-1$$ denotes the number of consecutive time slots for which the system is in an erroneous state. Furthermore, the transition probability [TeX:] $$P_{i, i+1}$$ is defined as [TeX:] $$P_{i, i+1}=\operatorname{Pr}\left[C_E(t+1)=\right.\left.i+1 \mid C_E(t)=i\right].$$ For the state-aware randomized stationary policy, this transition probability is given by

⁴ A similar metric was defined first in [36] and then in [37].

where [TeX:] $$\operatorname{Pr}\left[C_E(t)=i\right] \text { for } i=0$$ is equal to [TeX:] $$\operatorname{Pr}\left[C_E(t)=0\right]=\pi_{0,0}+\pi_{1,1} \text {, and for } i \geqslant 1 \text {, }$$ it is calculated as (see Appendix C)

where [TeX:] $$\pi_{i, j}, \forall i, j \in\{0,1\}$$ was given in Lemma 1. Now, using [TeX:] $$\operatorname{Pr}\left[C_E(t)=0\right]=\pi_{0,0}+\pi_{1,1}$$ and (13), the transition probability given in (12) can be written as

Using (13), we can calculate the average consecutive error [TeX:] $$\bar{C}_E$$ as

Note that the convergence conditions for the previous expression are [TeX:] $$\left|(1-p)\left(1-p_{\alpha_0^{\mathrm{s}}} p_{\mathrm{s}_0}\right)\right|\lt 1$$ and [TeX:] $$\left|(1-q)\left(1-p_{\alpha_1^{\mathrm{s}}} p_{\mathrm{s}_1}\right)\right|\lt1.$$

In the following, we define the consecutive error violation probability metric as the percentage of time during which the system remains in an erroneous state for more than n consecutive time slots. Therefore, using the expression given in (13) and Lemma 1, we can write

where n in (16) is finite.

B. Importance-Aware Consecutive Errors

In this section, we introduce a new timing-aware error metric as a means to capture the significance of a particular erroneous action at the receiver side. For that, we propose the importance-aware consecutive error metric, which is defined as the number of consecutive time slots that the system is in an particular erroneous state. Here, we assume that the system is in an erroneous action when the state of the source is [TeX:] $$1(X(t)=1) \text {, }$$ and the reconstructed source is in state [TeX:] $$0(\hat{X}(t)=0)$$. Now, let [TeX:] $$S(t) \neq 0$$ denote that the system is in the mentioned erroneous state at time slot t, while the synced state of the system is denoted by S(t) = 0. We also define [TeX:] $$C_S(t)$$ as the consecutive error at time slot t when the system is in the mentioned particular erroneous state. Now, we can define the state evolution of consecutive error as follows

Using (17), we define the transition probability of [TeX:] $$C_S(t)$$ as

Now, using the similar procedure presented in Section V-A, one can obtain [TeX:] $$\operatorname{Pr}\left[C_S(t)=i\right]$$ as follows

where [TeX:] $$\pi_{0,0} \text { and } \pi_{1,0}$$ are given in Lemma 1. Now, using (19) we can write (18) as

Remark 3. The transition probability [TeX:] $$P_{i, 0}^S(i \geqslant 0),$$ is defined as [TeX:] $$P_{i, 0}^S=1-P_{i, i+1}^S .$$

Now, using (19) and Lemma 1, the average importanceaware consecutive error, [TeX:] $$\bar{C}_S,$$ can be obtained as

where [TeX:] $$\Phi\left(p_{\alpha_0^{\mathrm{s}}}, p_{\alpha_1^{\mathrm{s}}}\right)$$ is given in (7).

Remark 4. We would like to emphasize that since the metric of importance-aware consecutive errors considers exclusively the scenario of a specific error, this metric needs to be studied in combination with another error metric, for instance, considering a constrained optimization problem or combining the metrics to form error vectors. This will become prominent in the numerical results section.

VI. OPTIMIZATION PROBLEM

In this section, our objective is to find an optimal stateaware randomized stationary sampling policy, which minimizes the average cost of actuation error subject to a timeaveraged sampling cost constraint. Here, we assume that each attempted sampling has a sampling cost δ, and that the timeaveraged sampling cost cannot exceed a certain threshold [TeX:] $$\delta_{\max }.$$ Therefore, the optimization problem is formulated as

where the constraint given in (22b) can be written as

Now, using (9) and (23), the optimization problem can be simplified as

where [TeX:] $$\eta=\delta_{\max } / \delta, \Psi\left(p_{\alpha_0^{\mathrm{s}}}, p_{\alpha_1^{\mathrm{s}}}\right)=C_{0,1} p_{\alpha_1^{\mathrm{s}}} p_{\mathrm{s}_1}\left(1-p_{\alpha_0^{\mathrm{s}}} p_{\mathrm{s}_0}\right)+C_{1,0} p_{\alpha_0^{\mathrm{s}}} p_{\mathrm{s}_0}\left(1-p_{\alpha_1^{\mathrm{s}}} p_{\mathrm{s}_1}\right),$$ and [TeX:] $$\Phi\left(p_{\alpha_0^{\mathrm{s}}}, p_{\alpha_1^{\mathrm{s}}}\right)$$ is given by (7).

To solve this optimization problem, we first note that the objective function in (24a) is decreasing with [TeX:] $$p_{\alpha_0^{\mathrm{s}}},$$ i.e., [TeX:] $$\frac{\partial P_E^C}{\partial p_{\alpha_0^{\mathrm{s}}}}\lt 0,$$ when

Also, the objective function is decreasing with [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ when

Based on (25) and (26), we consider two cases: one with [TeX:] $$p C_{1,0} \geqslant q C_{0,1}$$ and one other with [TeX:] $$p C_{1,0}\lt q C_{0,1}$$.

1) When [TeX:] $$p C_{1,0} \geqslant q C_{0,1}$$: In this case, we can always find a probability [TeX:] $$p_{\alpha_0^{\mathrm{s}}} \in[0,1]$$ that satisfies the condition given in (26). Therefore, since [TeX:] $$p_{\alpha_0^{\mathrm{s}}} \geqslant 0,$$ using (26), the objective function has its minimum value when [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ is maximized. Now, using the constraint given in (24b), the maximum value of [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ is

Using (27), the optimization problem can be written as

where [TeX:] $$F, G, p_{\alpha_0^{\mathrm{s}}}^{\mathrm{LB}} \text {, and } p_{\alpha_0^{\mathrm{s}}}^{\mathrm{UB}}$$ are given by

where [TeX:] $$A_i \text {, and } B_i \forall i \in\{1,2,3\} \text {, }$$ are given by

To determine the value of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ that minimizes the objective function in (28a), we need to calculate the critical points of the objective function within the interval [TeX:] $$\left[p_{\alpha_0^{\mathrm{s}}}^{\mathrm{LB}}, p_{\alpha_0^{\mathrm{s}}}^{\mathrm{UB}}\right].$$ When [TeX:] $$\left(2 A_1 B_3-2 A_3 B_1\right)^2 \geqslant 4\left(A_1 B_2-\right.\left.A_2 B_1\right)\left(A_2 B_3-A_3 B_2\right),$$ one can obtain the critical points of the objective function by taking the first derivative [TeX:] $$\frac{\partial}{\partial p_{\alpha_0^s}}\left(\frac{F}{G}\right)=0$$ as⁵

⁵ When Δ in (32) is negative, the optimal value of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ that minimizes the objective function in (28a) is equal to [TeX:] $$p_{\alpha_0^{\mathrm{s}}}^{\mathrm{LB}}$$ if [TeX:] $$A_1 B_2\gt A_2 B_1$$ and [TeX:] $$p_{\alpha_0^{\mathrm{s}}}^{\mathrm{UB}}$$ if [TeX:] $$A_1 B_2\lt A_2 B_1.$$

where [TeX:] $$A_i, \text { and } B_i$$ are given in (30) and Δ can be written as

Note that we consider only the value of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ in (31) within the interval [TeX:] $$\left[p_{\alpha_0^{\mathrm{s}}}^{\mathrm{LB}}, p_{\alpha_0^{\mathrm{s}}}^{\mathrm{UB}}\right].$$ Now, we evaluate the objective function at the critical points, as well as at points [TeX:] $$p_{\alpha_0^{\mathrm{s}}}^{\mathrm{LB}} \text { and }p_{\alpha_0^{\mathrm{s}}}^{\mathrm{UB}}$$. The minimum value of the objective function within the given interval corresponds to the smallest value. After determining the value of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ that minimizes the objective function, we can calculate [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ by utilizing the expression given in (27). We note that the values of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ and [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ obtained by solving the optimization problem in (28), are the optimal values of the sampling probabilities when [TeX:] $$p_{\alpha_1^{\mathrm{s}}} \geqslant \frac{p C_{1,0}-q C_{0,1}}{p_{\mathrm{s}_1}\left(p C_{1,0}+(1-q) C_{0,1}\right)}.$$ Otherwise, the optimal values of sampling probabilities [TeX:] $$p_{\alpha_0^{\mathrm{s}}} \text { and } p_{\alpha_1^{\mathrm{s}}}$$ are given by

This is because using (25), as [TeX:] $$p_{\alpha_1^{\mathrm{s}}}\lt \frac{p C_{1,0}-q C_{0,1}}{p_{\mathrm{s}_1}\left(p C_{1,0}+(1-q) C_{0,1}\right)},$$ the objective function in (24a) is increasing with [TeX:] $$p_{\alpha_0^{\mathrm{s}}}.$$ Therefore, the optimal values of the sampling probabilities are given by (33).

Remark 5. When [TeX:] $$p C_{1,0} \geqslant q C_{0,1}$$ and [TeX:] $$p_{s_1}\lt \frac{p C_{1,0}-q C_{0,1}}{p C_{1,0}+(1-q) C_{0,1}},$$ we cannot find a probability [TeX:] $$p_{\alpha_1^s} \in[0,1]}$$ that satisfies the condition given in (25). Therefore, in that case, the optimal values of [TeX:] $$p_{\alpha_0^s} \text { and }p_{\alpha_1^s}$$ that minimize the objective function in (24a) are given by [TeX:] $$p_{\alpha_0^s}^*=0 \text { and } p_{\alpha_1^s}^*=\min \{1, \eta(p+q) / p\} \text {. }$$

2) When [TeX:] $$p C_{1,0}\lt q C_{0,1}$$ : In this case, since [TeX:] $$p_{\alpha_1^s} \geqslant 0,$$ using (25), as [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ increases, the objective function in (24a) decreases. Using the constraint in (24b), the maximum value of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ is given by

Now, using (34), the optimization problem given in (24) is simplified as

where [TeX:] $$H, K, p_{\alpha_1^{\mathrm{s}}}^{\mathrm{LB}} \text {, and } p_{\alpha_1^{\mathrm{s}}}^{\mathrm{UB}}$$ are given by

Similar to the case when [TeX:] $$p C_{1,0} \geqslant q C_{0,1},$$ we can obtain [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ that minimizes the objective function in (35a) by calculating its critical points within the interval [TeX:] $$\left[p_{\alpha_1^{\mathrm{s}}}^{\mathrm{LB}}, p_{\alpha_1^{\mathrm{s}}}^{\mathrm{UB}}\right] .$$ Then, we obtain [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ using (34). We can similarly prove that when [TeX:] $$p_{\alpha_0^{\mathrm{s}}} \geqslant \frac{q C_{0,1}-p C_{1,0}}{p_{\mathrm{s}_0}\left(q C_{0,1}+(1-p) C_{1,0}\right)}, p_{\alpha_0^{\mathrm{s}}} \text { and } p_{\alpha_1^{\mathrm{s}}}$$ derived by solving the optimization problem in (35), are the optimal values of the sampling probabilities. Otherwise, the optimal values of the sampling probabilities are equal to [TeX:] $$p_{\alpha_0^s}^*=\min \{1, \eta(p+q) / q\} \text { and } p_{\alpha_1^s}^*=0.$$

Remark 6. In what follows, RS and RSC policies are the abbreviations for the state-aware randomized stationary policy and the state-aware randomized stationary policy in the constrained optimization problem, respectively.

VII. SIMULATION RESULTS

In this section, we validate our analytical results and evaluate the performance of the sampling policies in terms of timeaveraged reconstruction error and the average cost of actuation error under various system parameters. In the uniform policy, a sample is acquired every 5 time slots. Simulation results are obtained averaging over [TeX:] $$10^7$$ time slots.

In Tables I through VI, we illustrate the minimum average cost of actuation error when [TeX:] $$C_{0,1}=1, C_{1,0}=2$$ under a sampling cost constraint for [TeX:] $$\eta=0.5$$, and various values of p, q, [TeX:] $$p_{\mathrm{s}_0}, \text { and } p_{\mathrm{s}_1}$$. As seen in these Tables, when [TeX:] $$p C_{1,0} \geqslant q C_{0,1},$$ the average cost of actuation error has its minimum value when [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ is greater than [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$. Otherwise, the minimum average cost of actuation error occurs for [TeX:] $$p_{\alpha_0^{\mathrm{s}}}\gt p_{\alpha_1^{\mathrm{s}}}.$$ Note also that for the lower successful probabilities, the minimum average cost of actuation error occurs at small values of [TeX:] $$p_{\alpha_0^s}^* \text { and }p_{\alpha_1^s}^*,$$ where [TeX:] $$p C_{1,0} \geqslant q C_{0,1}$$ and [TeX:] $$q C_{0,1}\gt p C_{1,0},$$ respectively. Furthermore, we observe that the optimal RSC policy exhibits superior performance than the semantics-aware policy under the conditions given in Remark 1, for both slow and fast changing the information source. Otherwise, when Remark 1 is not satisfied, the semantics-aware policy performs better only when the source is slowly evolving. Note that the optimal values in red color for the semantics-aware, change-aware, and RS policies are obtained for values of p, q, [TeX:] $$p_{\mathrm{s}_0}, \text { and } p_{\mathrm{s}_1}$$ that do not satisfy the constraint requirement. This means that in the unconstrained scenario, the performance of the optimal RS policy is either better or the same as that of the semantics-aware policy. However, in this case, the optimal solution for the RS policy is to sample and transmit at most of the time slots, resulting in an excessive amount of samples being generated.

The optimal values for sampling probabilities in the unconstrained scenario are shown in Tables III and VI. As observed in these Tables, for all values of p and q, we have [TeX:] $$p C_{1,0} \geqslant q C_{0,1} .$$ Consequently, the optimal value of [TeX:] $$p_{\alpha_1^s}^*$$ is 1. Furthermore, for values of p and q where [TeX:] $$p_{\mathrm{S}_1}\lt \frac{p C_{1,0}-q C_{0,1}}{p C_{1,0}+(1-q) C_{0,1}},$$ the optimal value of [TeX:] $$p_{\alpha_0^s}^*$$ is 0; otherwise, [TeX:] $$p_{\alpha_0^s}^*=1.$$ This implies that when the success probability of a state is low, the optimal solution is to refrain from sampling for the state that causes the less important error in terms of actuation, while for a higher success probability, the optimal solution is to always perform sampling.

The performance of the optimal state-aware randomized stationary policy in terms of time-averaged reconstruction error as a function of η for [TeX:] $$p_{\mathrm{s}_0}=0.5, p_{\mathrm{s}_1}=0.6,$$ and different values of p, and q is shown in Tables VII–X. We observe that the time-averaged reconstruction error has a smaller value as η increases. This is because η is the threshold of the total timeaveraged sampling cost, thus a higher value of η results in higher sampling probabilities which in turn decreases the timeaveraged reconstruction error, as demonstrated by Tables VIII and X, q > p and [TeX:] $$p_{\mathrm{S}_0}\gt \frac{q-p}{1+q-p}.$$ Hence, the minimum timeaveraged reconstruction error in the unconstrained scenario is obtained with [TeX:] $$p_{\alpha_0^s}^*=1 \text { and } p_{\alpha_1^s}^*=1.$$

Figs. 4 and 5 show the average consecutive error contour plots as a function of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ and [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ for p > q, considering the slow and rapid changes of the source, respectively. As illustrated in Fig. 4, when the source changes slowly, the minimum average consecutive error occurs at high values of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ and [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$. In addition, as observed in Fig. 5, when the source changes rapidly and success probabilities are low, the average consecutive error decreases with a high value of [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ and a low value of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$. Furthermore, when success probabilities are high, the average consecutive error has its minimum value as [TeX:] $$p_{\alpha_0^{\mathrm{s}}} \text { and } p_{\alpha_1^{\mathrm{s}}}$$ increase. Also, note that these figures can be used to obtain the optimal values of sampling probabilities. Another noteworthy result is that as the success probabilities increase, we can achieve a comparable average consecutive error with smaller sampling probabilities compared to situations where the success probabilities are lower. For example, when p = 0.3 and q = 0.2, with [TeX:] $$p_{\mathrm{s}_0}=0.2 \text { and } p_{\mathrm{s}_1}=0.3,$$ the minimum average consecutive error is approximately 0:65, which is achieved by setting [TeX:] $$p_{\alpha_0^{\mathrm{s}}}=1 \text { and } p_{\mathrm{s}_1}=1.$$ However, for [TeX:] $$p_{\mathrm{s}_0}=0.7 \text { and } p_{\mathrm{s}_1}=0.8,$$ the similar average consecutive error value can be obtained by using [TeX:] $$p_{\alpha_0^{\mathrm{s}}}=0.2 \text { and } p_{\alpha_1^{\mathrm{s}}}=1 \text {. }$$

The average importance-aware consecutive error as a function of [TeX:] $$p_{\mathrm{s}_0} \text { and } p_{\mathrm{s}_1}$$ for slowly and rapidly changing information source, and selected values of success probabilities, is presented in Figs. 6 and 7. In this analysis, we focus on the particular erroneous state where [TeX:] $$X(t)=1 \text { and } \hat{X}(t)=0.$$ As seen in these figures, the average importance-aware consecutive error is minimized when [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ is at its maximum and [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ is at its minimum. The reason behind this is that when we have poor channel conditions, successful transmission of the less important state can have a negative impact since when the source will transit to the important state, the destination may miss that transition due to a potentially unsuccessful transmission. Thus, in that case, it may be preferable to avoid performing sampling in the less important state. To avoid such cases, this metric has to be considered and optimized in combination with other error metrics, such as the time-averaged reconstruction error. This is because a decrease in [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ may lead to an increase in the time-averaged reconstruction error, depending on the other system parameters. This increase can have significant consequences for the overall performance of such systems. Therefore, it is crucial to consider the interplay between the average importance-aware consecutive error and the time-averaged reconstruction error. Tables XI and XII illustrate the average importance-aware consecutive error and time-averaged reconstruction error as a function of [TeX:] $$p_{\alpha_0^{\mathrm{s}}}$$ and [TeX:] $$p_{\alpha_1^{\mathrm{s}}}$$ for [TeX:] $$p_{\mathrm{s}_0}=0.4, p_{\mathrm{s}_1}=0.7,$$ p = 0.5, and q = 0.9. Here we intend to find sampling probabilities [TeX:] $$p_{\mathrm{s}_0} \text{ and } p_{\mathrm{s}_1}$$ to keep the average importance-aware consecutive error and time-averaged reconstruction error below predefined thresholds [TeX:] $$I_1 \text{ and } I_2,$$ respectively. For example, we assume that the thresholds are equal to [TeX:] $$I_1=0.1 \text{ and } I_2=0.3.$$ As seen in these tables, we can achieve our purpose by setting [TeX:] $$p_{\alpha_0^{\mathrm{s}}}=1 \text { and } p_{\alpha_1^{\mathrm{s}}}=0.9 \text{ or } 1.$$ However, when [TeX:] $$I_1=0.1 \text{ and } I_2=0.2,$$ we cannot find sampling probabilities [TeX:] $$p_{\alpha_0^{\mathrm{s}}} \text { and } p_{\alpha_1^{\mathrm{s}}}$$ to fulfill our objective. While if we only consider the average importance-aware consecutive error metric, we could achieve our purpose by setting [TeX:] $$p_{\alpha_0^{\mathrm{s}}}=0.1 \text { and } p_{\alpha_1^{\mathrm{s}}}=1 \text {. }$$

VIII. CONCLUSIONS

In this work, we considered a time slotted communication system where sampling and transmission over a wireless erasure channel is performed in order to track a two-state Markov process. We proposed a state-aware randomized stationary, which considers varying sampling and success probabilities for different states of the source. We then analyzed the system performance in terms of set of metrics, namely timeaveraged reconstruction error, average cost of actuation error, consecutive error, and importance-aware consecutive error. Furthermore, we cast and solved the optimization problem of minimizing the average cost of actuation error while keeping the time-averaged sampling cost constraint less than a given threshold. Our results illustrated that the optimal state-aware randomized stationary policy exhibits superior performance compared to other state-of-the-art policies when there is a constraint on the sampling cost, in particular when the source varies quickly. Specifically, it performs better for fast changes in the source and can also perform well for slow changes under certain conditions.

APPENDIX A

PROOF OF LEMMA 1

To obtain [TeX:] $$\pi_{i, j}$$ we depict the two-dimensional DTMC describing the joint status of the system regarding the current state at the original source, i.e., [TeX:] $$(X(t), \hat{X}(t))$$ in Fig. 8, where [TeX:] $$P_0, P_1, P_2 \text {, and } P_3$$ are given by

Now, using Fig. 8 and (38), we can obtain state stationary [TeX:] $$\pi_{i, j}, \forall i, j \in\{0,1\},$$ as follows

where [TeX:] $$\Phi\left(p_{\alpha_0^{\mathrm{s}}}, p_{\alpha_1^{\mathrm{s}}}\right)$$ is given in (7). For the change-aware policy, (39) can be written as

Furthermore, for the semantics-aware policy [TeX:] $$\pi_{i, j}$$ is given by

APPENDIX B

THREE-STATE DTMC INFORMATION SOURCE

For a three-state DTMC information source model depicted in Fig. 9, the stationary distribution [TeX:] $$\pi_{i, j}$$ for the state-aware randomized stationary policy is given by

where [TeX:] $$Z_1 \text { and } Z_2$$ in (42) to (50) are given by

APPENDIX C

PROOF OF (13)

[TeX:] $$C_E(t)=i(i \geqslant 1)$$ means that the system was in a synced state at time slot t-i, and it has been in an erroneous state from time slots t - i + 1 to t. Therefore, to obtain [TeX:] $$\operatorname{Pr}\left[C_E(t)=i\right],$$ we need to calculate

where the first conditional probability in (53) can be written as

Similarly, one can obtain the second conditional probability in (53) as

Now, using Lemma 1, (54) and (55), we can write (53) as