http://www.BillHowell.ca/Neural nets/Paper reviews/5_NN math defs, symbols, nomenclature - time-scale systems.txt
www.BillHowell.ca 29Jan2018 initial taken from :
http://www.BillHowell.ca/Neural nets/Paper reviews/5_NN math defs, symbols, nomenclature - NN stability with delays, impulses.txt
*****************************
>>>>> "Flat-line" UNICODE nomenclature :
This style of writing expressions allows the use of any UNICODE-enabled text editor without need for subscripts or superscripts. It also facilitates faster copy-paste re-use of expression. Furthermore, although not used for this paper, the intent is to facilitae used of expressions in a symbolic processor (I've only used this in the past for very limited capablities). Admittedly, it is a nuisance to adapt and lacks the clarity of standard formatting.
Various notations may be combined...
A variable [scalar, vector, matrix] for use in this list of notations
A(i,j) (i,j)th component of A
A(i,j) (i,j)th component of A, notation just for simple indices
A_T transpose of A
|A| absolute value of matrix A (each element)
||A|| [Euclidean, spectral] norm of A ||A||2
p3h0.25 ||·|| denotes the Euclidean norm.
||A||_p = ( sum(i from 1 to n : |A(i)|^p )^(1/p) ) denotes the p-norm of vector z
A_dot time derivative of A, same as d[dt: A]
A_til A tilde over
A_ovr A overlined
A_und A underlined
A_hat A with ^ above it
A_sup(x) A superscript x - to avoid confuxion with A^x
A_sub(x) A subscript x - as distinct from A(x)
A^x A to the power of x
λ max ( λ min ) refers to maximum (minimum) eigenvalue
t_minus t approached from t < t_minus
t_plus t approached from t_plus < t
Indexing :
yi y(i), simple concatenation (might be confused with y*i)
y(i+1) array type notation
Note the more clear & easy to read indexing used in this review for mixing indexes and t or other variables
... example ai(t) -> a(i,t)
Inequalities (for faster typing, symbolic processing)
≤ <=
≥ >=
≠ !=
[super, sub]-script symbols
"_cup" denotes variable values during ??? eg p4L23 Eq (9d)
"_ovr" denotes variable values during times s ∈ [t_til(0),0]??? eg ξ_ovr(t)
"_til" denotes variable values during ??? eg p4L38
Un-numbered [equations, inequalities] in the paper are denoted by their location, enclosed by parenthesis, eg :
* y(t) ≤ ρ*sup[s∈[t−T,t]: y(s)]
means the above equation on page 5, Line 42.
Reviewer equations are denoted by some [equation, section, subsection, whatever] and index, enclosed by parenthesis, eg :
(R.10a.5) z(t**) ≤ d/c*z(t*)*e^(-θ*ln(c)/T) , ∀t ≥ t0
Calculus :
d[dt: x] total derivative of x with respect to t
dp[dt: x] partial derivative of x with respect to t
d^n[dt^n: x] nth total derivative of x with respect to t
dp^n[dt^n: x] nth partial derivative of x with respect to t
∫[ds: f] indefinite integral of f with respect to s
∫[ds, a to b: f] definite integral of f with respect to s, from a to b
Other :
sum[i=1 to n: f(n,t)] sum of a series
Π[tt_plus : (r(t,t) - r(s,t))/(t - s)) }
D_minus D_minus(t) = lim{s->t_minus: (r(t,t) - r(s,t))/(t - s)) }
******* denotes start/end of topics & sub-topics
+-----+ denotes sub-steps in a [proof, development]..
denote [start,end] of checks on specific steps by the reviewer (me), on expression (1) in the example below :
>>>>> check (?) :
>> ???? check (?) unfinished
<<<<< end check.
>> start of short reviewer comments
<< end of short reviewer comments (Often this is not used as only one or two lines of comment are involved
>>** comment for extraction to main part of review, Section "C6. MATH CHECKS - step-by-step"
***************************
>>>>> Symbols
+-----+
first characters of line - indicates definition used in a similar paper, which makes for interesting contrasts,
plus confirmation of a usage of variable symbols that is similar to other papers in the area
(*) taken from earlier papers, mostly for NN stability Theorems
p2h0.8 (example) - taken from "180207 conference paper review - math only.txt"
kwrite editor, regular expression -> Search : ^\(p[0-9]*L[0-9]*\), Replace : \(171127\)
special contexts :
- normally, I will use a Latin letter in place of Greek or some Unicode if that letter is not used elsewhere
(for ease of typing)
- Note that the authors use symbols in a manner consistent with other papers in the area, of course making this paper easier to follow, and to identify subtle differences in assumptions etc.
+-----+
General list : normally in alphabetical order, except some "collections" of themes (start/end with +--+)
+--+
Metric spaces, sets (hexa refers to U+hexa)
LibreWrite (Ctrl+Shift)+U+hexa
kwrite text editor enter into LibreWrite, copy to kwrite?
* 𝕁 subset of ℝ^n
* ℝ, real numbers
* ℝ+, non-negative real numbers
p3h0.25 ℝ denotes the real number set
p3h0.25 ℝ+ denotes the nonnegative real number set
* ℝ^n n-dimensional Euclidean spaces
* ℝ^(n×n) n×n real matrix
p2h0.75 𝓡 and 𝓡+ denote the set of regressive and positive regressive functions;
* 𝕊 subset of ℝ
p2h0.33 𝕋 represents a time scale which is an arbitrary nonempty closed subset of the real set ℝ.
p3h0.3 min{𝕋} = 0 Without other statements, we always assume this
p3h0.75 𝕋aux ∆= {t(k,2)}|k=1 to ∞ an auxiliary discrete time scale
where
p3h0.75 t(i,2) is the i-th right-scattered point of 𝕋.
* ℤ positive integers
p3h0.25 ℤ denotes the integer number set
+--+
Latin symbols sorted in Latin alphabetical order
parameters of inertial NN (1), real-valued continuous functions, i ∈ N
* a(i,t) > 0
* b(i,t) > 0
* c(i,j,t) connection weights without delays
* d(i,j,t) connection weights with delays
* A(t) = diag[α(1,t),α(2,t),...,α(n,t)]_T
* B(t) = diag[β(1,t),β(2,t),...,β(n,t)]_T
* C(t) = ( c(i,j,t) )^(n*n)
* D(t) = ( d(i,j,t) )^(n*n)
p4h0.65 A = max[t≤T,t ∈ ℝ+(𝕋) : {G(t)} ]
p4h0.65 B = max{M, A},
* C^+(S,J) = {z : S → J is continuous and its upper right Dini derivative D+z exists }
... for any interval S ⊆ ℝ, J ⊆ ℝ^n
* D^+[dz: f(z)] ∆= lim[sup[h -> 0+: (f(z+h) - f(z))/h], Dini derivative of z
... https://en.wikipedia.org/wiki/Dini_derivative
p3h0.0 c a regressive constant
p2h0.75 Crd = Crd(𝕋) = Crd(𝕋,ℝ) denotes the set of rd-continuous functions
p2h0.85 e(p,t,s) = exp( ∫[∆τ from s to t: ξ(µ(τ),p(τ)) ] ) for p ∈ 𝓡, s,t ∈ 𝕋^k
... exponential function definition
p3h0.0(a) e(c,t,0) = exp(c*t) for any t ∈ ℝ, 𝕋 = ℝ timescale-type exponential function
p3h0.0(b) e(c,t,0) = (1 + c*h)^(t/h) for any t ∈ h*ℤ, 𝕋 = h*ℤ timescale-type exponential function
p2h0.85 e(p,t,s) = exp( ∫[∆τ from s to t: ξ(µ(τ),p(τ)) ] ) for p ∈ 𝓡, s,t ∈ 𝕋^k exponential function definition
* f = (f1,···,fn) ∈ C(ℝn,ℝn) is the activation function
* f(j,ξ(j,t)) represents the neuron activation function of the jth neuron at time t
p1h0.75 f : 𝕋 × ℝ^n → ℝ^n is a rd-continuous function satisfying f(t,0) = 0 for all t ∈ 𝕋.
p2h0.75 fσ(t) = f(σ(t))
p2h0.75 f∆(t) denotes the ∆-derivative of f at t ∈ 𝕋;
* |f(j,u) - f(j,v)| ≤ l(j)*|u - v| ∀u,v ∈ R, j ∈ N maximum |"gradient"| sort-of
p4h0.6 G(t) = ∫[∆τ, from 0 to t : g(τ) ]
p3h0.5 g : ℝ+ → ℝ a regressive function
* J(i,t) denotes the neuron of an external input on the ith neuron
* J ⊆ ℝ^n interval
* k ∈ N0
* K = {f ∈ C+(ℝ,ℝ+) | f(t) ≤ 1, if t ≤ 0; f(t) ≥ 1, if t > 0 }
p1h0.95 k ∈ ℤ discrete time index
* L = diag(l1,···,ln)
* L = diag{l,l,...,l}, where l = max{1≤j≤n: l(j) }
* l(j) are positive constants
* l = max[1≤j≤n: l(j) ]
* M ≥ 0 constant
* M > 0 constant Eq (8)
p4h0.65 M < 0 constant
* N = {1,2···},
* N = {1,2,···,n} total number of neurons
* N0 = N ∪ {0}
* PC(J,_ovr,∆) = {φ : J_ovr → ∆ is continuous everywhere except at finite
... number of points t, at which φ(t+), φ(t−) exist and φ(t+) = φ(t)}.
... assume "PC" stands for "Pulse Condition" or something like that
* PC(S,J) = {z : S → J is continuous everywhere except at finite number of points t
... at which z(t+), z(t−) exist and z(t−) = z(t) }
p2h0.8 ⊖p = -p(t) /(1+µ(t)p(t)) p ∈ 𝓡+
p2h0.8 p⊖q = (p(t)−q(t))/(1+µ(t)q(t)) p, q ∈ 𝓡+
* R(l) I, l ≥ 1, is impulsive matrix
* R(t) = diag[γ(1,t),γ(2,t),...,γ(n,t)]_T but γ(1,t) = γ(1t)!?!
p2h0.75 𝓡 and 𝓡+ denote the set of regressive and positive regressive functions
p2h0.75 rd-continuous = right dense continuous (http://campus.mst.edu/adsa/contents/v6n2p1.pdf)
* s ∈ [t_til(0),0], for Eq (3), (p3L0) S ⊆ ℝ interval
* t ≥ t0 , denotes the reset rate of neurons
* t_til(0) = inf{over t ∈ [0,∞): t − τ(t) } often use s ∈ [t_til(0),0] Eq (2)
* t∗ = t∗ + l*T + θ, where θ ∈ [0, T )
* t* = ∈ [0,t1)
* t∗∗ ∈ [tK,t(K+1))
* {tk}_sup(∞)_sub(k=1) is the impulsive time sequence
* tk impulse time, satisfy 0 < t1 < t2 < ... < tk → ∞ as k → ∞
* T is the length of a time sub-interval within t* -> t**
... such that T(1) ≤ t(k+1) − t(k) ≤ T(2) with T(2) ≥ T(1) > 0
p3h0.65 UUB ultimate upper-bound
p3h0.65 UUB = α^(−1)(V(0,x0)*e^λ ) by Theorem 1
* V(t) impulsive differential delay inequality function Eq (11a)
... while not introduced as a Lyapunov functional, as below with V_Lyap(t), that is what is used
p3h0.5 V : ℝ+(𝕋) × ℝ^n → ℝ+ with V (t,0) = 0 for all t ∈ ℝ+(𝕋)
... an rd-continuous differentiable function
p3h0.6(ii) V∆(t,x) ≤ g(t)*V(t,x) for t ∈ ℝ+(𝕋), with ∫[∆t, from 0 to ∞: g(t)*∆t] ≤ λ
* V_Lyap(t) Lyapunov function Eq (29)
* V_til(0) = sup[s ∈ [t_til(0),0]: V(s)], where V_ovr(t) ≠ 0, for Eq (14)
* vk ≥ 1 constant between impulses for Eq (11)
* W(t) == V(t)*ω(t) ≤ Π[k=1 to ι-1: vk*V_til(0) ] Eq (14)
... note "==" is used for U+2250, which doesn't display in this text editor
* X(t) = (ξ(t)_T, η(t)_T)_T solution of (7)
* Y(t) = (ξ_ovr(t)_T, η_ovr(t)_T)_T solution of (7)
* x = (x1,···,xn) T ∈ ℝ^n is the state vector of neurons
p1h0.75 x(t) ∈ ℝ^n is the state of the system,
p1h0.6(1) x∆(t) = f (t,x(t)), t ∈ 𝕋, timescale-type time-varying system
p1h0.8(3a) d[dt: x(t)] = f(t,x(t)), t ∈ ℝ,
p1h0.95(4a) x(k+1) = x(k) + f(k,x(k)), k ∈ ℤ,
* x(i,t) = ξ(i,t) - ξ_ovr(i,t)
* y(i,t) = η(i,t) - η_ovr(i,t)
* x(t) = (x(1,t),x(2,t),...,x(n,t))_T
* y(t) = (y(1,t),y(2,t),...,y(n,t))_T
* z(t) = (z(1,t),z(2,t),...,z(2*n,t))_T = (x(t)_T,y(t)_T] )_T
p3h0.3 = inf{ x_til ∈ 𝕋|x_til > x} For any x ∈ ℝ
Greek symbols sorted in greek alphabet order
(variables for which I used similar Latin symbols instead of Greek are listed under Latin symbols)
* α(i,t) = γ(i)*{a(i,t) - γ(i,t)} - b(i,t) Eq (7b)
p3h0.5 α a K-class function
* β(i,t) = a(i,t) - γ(i,t) Eq (7b)
* γk positive constant
* γ(i,t) -> seems to be an incorrect notation (1st used Eq (4))
* γ(i) - constant term (reviewer substitute for γ(i,t) : 1st used Eq (4))
p2h0.75 Δ derivative; f∆(t) denotes the ∆-derivative of f at t ∈ 𝕋;
* δk constant
* ζk ∈ [−2,0] represent the strength of impulses
* η(t) = { η(1,t),η(2,t),...,η(i,n) }_T
* η(i,t) expression related to d[dt: ξ(i,t)] and ξ(i,t) used to simplify expressions, defined by (4)
* η(i,tk+) = lim[t→tk+0 : η(i,t)], i.e., the solution η(i,t) is right continuous at impulse point tk
* θ ∈ [0,T) (p14L5), and as ε≥0 (p4L37), this implies that θ≥0?
* θ(s) = { φ(s)_T, ψ(s)_T }
* Θ(s) = { φ(1,s),φ(2,s),...,φ(2*n,s) }_T = (φ_cup(s)_T , ψ_cup(s)_T)_T for s ∈ [t_til(0),0] for Eq (10)
* ι ∈ ℤ, eg t ∈ [0,tι)
* λ > 0 constant Eq (8)
p2h0.75 µ denotes the graininess function
* ξ(t) = { ξ(1,t),ξ(2,t),...,ξ(i,n) }_T
* ξ(i,t) denotes the state vectors of the ith neurons at time t,
... solution is right continuous at impulse point tk
... second-order derivative of ξ(i,t) is called an inertial term of system (1)
* ξ(i,tk+) = lim[t->tk+0: ξ(i,t)] = ξ(i,tk),
... i.e., the solution ξ(i,t) is right continuous at impulse point tk
p2h0.9 ξ(h,z) = { log(1+zh)/h for h ≠ 0; z for h = 0 } cylinder transformation definition
* ∆ξ(i,tk) = ξ(i,tk+) − ξ(i,tk-) = δk*x(i,tk) Eq (7c)
* ξ(i,s) = φ(i,s), s ∈ [t_til(0),0], i ∈ N
* η(i,s) = ψ(i,s), s ∈ [t_til(0),0], i ∈ N
* Π(1,t) = max{ max[1≤i≤n: sum[j=1 to n: |c(j,i,t)|*l] + |α(i,t)| - γ(i,t) ], max[1≤i≤n: 1 - β(i,t) ] }
* Π(2,t) = { sum[i=1 to n: max[1≤j≤n: |d(j,i,t)| ] ] }*l
* ρ(t) ≥ 0 continuous function for Eq (11)
p2h0.75 ρ backward jump operator
* ς(t) ≥ 0 continuous function for Eq (11)
p2h0.75 σ denotes the forward jump operator
* τ(t) is the time-varying delay satisfying τ(t) ≤ τ_ovr with τ_ovr > 0,
* τ(t) ∈ PC(R+,R+) time delay satisfying t − τ(t) → ∞ as t → ∞ for Eq (1)
* φ(s) ∈ PC([-τ_ovr,0],ℝ^n) is the initial condition
* Φ(t) = { Φ(1,t),Φ(2,t),...,Φ(i,n) }_T
+--+
initial conditions of inertial NN (1), real-valued continuous functions, s ∈ [t_til(0),0], i ∈ N
* Φ(i,s) = ξ(i,s), s ∈ [t_til(0),0], i ∈ N p3L18
* φ(i,s) = d[ds : ξ(i,s)], s ∈ [t_til(0),0], i ∈ N p3L18
* Φ(s) = { φ(1,s),φ(2,s),...,φ(2*n,s) }_T = (φ_cup(s)_T , ψ_cup(s)_T)_T for s ∈ [t_til(0),0] for Eq (10)
* φ_cup(s) = { φ_cup(1,s),φ_cup(2,s),...,φ_cup(n,s) }_T for Eq (10)
+--+
* ψ(t) = { ψ(1,t), ψ(2,t),...,ψ(i,n) }_T for (7)
* ψ(s) = { Φ_ovr(s)_T, ψ_ovr(s)_T }_T initial conditions for (8)
* ψ_cup(s) = { ψ_cup(s),ψ_cup(s),...,ψ_cup(s) }_T initial conditions for (10)
* ω(t) ∈ K satisfying ω(t) → ∞ as t → ∞, Eqs (11)(12)
* ω_ovr(k) = { max |δk| + 1, |ζk| + 1 } Eq (28)
*******************************************
>>>>> MATH DEFINITIONS
+-----+
Definition 1. System (1) is said to be:
1. Bounded if for any initial value x0 , there exists a positive scalar δ = δ(x0 ) such that kx(t)k ≤ δ for all t ∈ T.
2. Lyapunov stable if for an arbitrary ε > 0, there exists δ(ε, t0 ) such that kx(0)k < δ ⇒ kx(t)k < ε, ∀t ∈ T.
3. Asymptotically stable if it is Lyapunov stable, and there is a constant c = c(x0 ) > 0 such that x(t) → 0 as t → ∞, for all kx0 k < c.
inf (infinum) inf = inf{ x_tild ∈ T| x_tild > x}
# enddoc