nolog ann_FF_grad_BP x x_t N W c af err_deriv_y ...trace call to operation ... ann_ff_grad_bp ...the arguments for the opform are ... X X_t N W C Af Err_deriv_y ... N 4 2 4 ... gage shape N 3 ... Layers := gage shape N 3 ... X 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... gage second shape X 4 ... Patterns := gage second shape X 4 ... N 4 2 4 ... 1 N +-+-----+ |1|4 2 4| +-+-----+ ... + 1 N 5 3 5 ... N_bias := + 1 N 5 3 5 ... N_bias 5 3 5 ... N_bias EACHLEFT reshape 0 +---------+-----+---------+ |0 0 0 0 0|0 0 0|0 0 0 0 0| +---------+-----+---------+ ... Outputs := N_bias EACHLEFT reshape 0 +---------+-----+---------+ |0 0 0 0 0|0 0 0|0 0 0 0 0| +---------+-----+---------+ ... Err_dz_deriv_af := 0 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... 0 * W +--------------+--------+ |0. 0. 0. 0. 0.|0. 0. 0.| |0. 0. 0. 0. 0.|0. 0. 0.| | |0. 0. 0.| | |0. 0. 0.| +--------------+--------+ ... Grad_e := 0 * W +--------------+--------+ |0. 0. 0. 0. 0.|0. 0. 0.| |0. 0. 0. 0. 0.|0. 0. 0.| | |0. 0. 0.| | |0. 0. 0.| +--------------+--------+ ... Patterns 4 ... tell Patterns 0 1 2 3 ... Layers 3 ... tell Layers 0 1 2 ... I 0 ... I = 0 l ... P 0 ... [ , P ] ?noexpr 0 ... X 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X | [ , P ] 1 0 0 0 ... Out_base := X | [ , P ] 1 0 0 0 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 0 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 1 0 0 0 ... Bias @ I link Out_base 1 1 0 0 0 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+---------+ |ann_log_activ|1 1 0 0 0| +-------------+---------+ ...the arguments for the opform are ... X ... X 1 1 0 0 0 ... -1 * X -1 -1 0 0 0 ... 1 + exp ( -1 * X ) 1.36788 1.36788 2. 2. 2. ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.731059 0.5 0.5 0.5 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.731059 0.5 0.5 0.5 ... I 0 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+-----+---------+ |0.731059 0.731059 0.5 0.5 0.5|0 0 0|0 0 0 0 0| +-----------------------------+-----+---------+ ... I 1 ... I = 0 o ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+-----+---------+ |0.731059 0.731059 0.5 0.5 0.5|0 0 0|0 0 0 0 0| +-----------------------------+-----+---------+ ... Outputs @ ( I - 1 ) 0.731059 0.731059 0.5 0.5 0.5 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0.731059 0.731059 0.5 0.5 0.5| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+-----------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) -0.0430426 0.399048 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) -0.0430426 0.399048 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base -0.0430426 0.399048 ... Bias @ I link Out_base 1 -0.0430426 0.399048 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+---------------------+ |ann_log_activ|1 -0.0430426 0.399048| +-------------+---------------------+ ...the arguments for the opform are ... X ... X 1 -0.0430426 0.399048 ... -1 * X -1 0.0430426 -0.399048 ... 1 + exp ( -1 * X ) 1.36788 2.04398 1.67096 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.489241 0.598459 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.489241 0.598459 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+---------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0 0 0 0 0| +-----------------------------+--------------------------+---------+ ... I 2 ... I = 0 o ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+--------------------------+---------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0 0 0 0 0| +-----------------------------+--------------------------+---------+ ... Outputs @ ( I - 1 ) 0.731059 0.489241 0.598459 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+--------------------------+ | 0.16807 -0.760828 0.764471|0.731059 0.489241 0.598459| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+--------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.208145 0.599259 0.547915 -0.126359 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.208145 0.599259 0.547915 -0.126359 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0.208145 0.599259 0.547915 -0.126359 ... Bias @ I link Out_base 1 0.208145 0.599259 0.547915 -0.126359 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+--------------------------------------+ |ann_log_activ|1 0.208145 0.599259 0.547915 -0.126359| +-------------+--------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.208145 0.599259 0.547915 -0.126359 ... -1 * X -1 -0.208145 -0.599259 -0.547915 0.126359 ... 1 + exp ( -1 * X ) 1.36788 1.81209 1.54922 1.57815 2.13469 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.551849 0.645487 0.633652 0.468452 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.551849 0.645487 0.633652 0.468452 ... I 2 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Layers 3 ... Layers - 1 2 ... reverse count ( Layers - 1 ) 2 1 ... I 2 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) l ... Err_deriv_y ann_d_sum_of_sqr ... I 2 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.551849 0.645487 0.633652 0.468452 ... rest Outputs @ I 0.551849 0.645487 0.633652 0.468452 ... P 0 ... [ , P ] ?noexpr 0 ... X_t 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X_t | [ , P ] 1 0 0 0 ... ( rest Outputs @ I ) X_t | [ , P ] +-----------------------------------+-------+ |0.551849 0.645487 0.633652 0.468452|1 0 0 0| +-----------------------------------+-------+ ... Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) +----------------+---------------------------------------------+ |ann_d_sum_of_sqr|+-----------------------------------+-------+| | ||0.551849 0.645487 0.633652 0.468452|1 0 0 0|| | |+-----------------------------------+-------+| +----------------+---------------------------------------------+ ...the arguments for the opform are ... Y T ... Y 0.551849 0.645487 0.633652 0.468452 ... T 1 0 0 0 ... Y - T -0.448151 0.645487 0.633652 0.468452 ...end of operation call ... apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) -0.448151 0.645487 0.633652 0.468452 ... Err_dz := apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) -0.448151 0.645487 0.633652 0.468452 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 2 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.551849 0.645487 0.633652 0.468452 ... rest Outputs @ I 0.551849 0.645487 0.633652 0.468452 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------------------------+ |ann_d_log_activ|0.551849 0.645487 0.633652 0.468452| +---------------+-----------------------------------+ ...the arguments for the opform are ... Y ... Y 0.551849 0.645487 0.633652 0.468452 ... Y 0.551849 0.645487 0.633652 0.468452 ... 1 - Y 0.448151 0.354513 0.366348 0.531548 ... Y * ( 1 - Y ) 0.247312 0.228834 0.232137 0.249005 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.247312 0.228834 0.232137 0.249005 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.247312 0.228834 0.232137 0.249005 ... Err_dz -0.448151 0.645487 0.633652 0.468452 ... Deriv_af 0.247312 0.228834 0.232137 0.249005 ... Err_dz * Deriv_af -0.110833 0.147709 0.147094 0.116647 ... Err_dz_deriv_af := Err_dz * Deriv_af -0.110833 0.147709 0.147094 0.116647 ... I 2 ... I - 1 1 ... Grad_e +--------------+--------+ |0. 0. 0. 0. 0.|0. 0. 0.| |0. 0. 0. 0. 0.|0. 0. 0.| | |0. 0. 0.| | |0. 0. 0.| +--------------+--------+ ... Grad_e @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Err_dz_deriv_af -0.110833 0.147709 0.147094 0.116647 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.489241 0.598459 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +------------------------------------+--------------------------+ |-0.110833 0.147709 0.147094 0.116647|0.731059 0.489241 0.598459| +------------------------------------+--------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) -0.0810253 -0.054224 -0.0663289 0.107984 0.0722653 0.0883978 0.107534 0.0719645 0.0880298 0.0852757 0.0570684 0.0698083 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) -0.0810253 -0.054224 -0.0663289 0.107984 0.0722653 0.0883978 0.107534 0.0719645 0.0880298 0.0852757 0.0570684 0.0698083 ... I 2 ... I - 1 1 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +--------------+-------------------------------+ |0. 0. 0. 0. 0.|-0.0810253 -0.054224 -0.0663289| |0. 0. 0. 0. 0.| 0.107984 0.0722653 0.0883978| | | 0.107534 0.0719645 0.0880298| | | 0.0852757 0.0570684 0.0698083| +--------------+-------------------------------+ ... I 1 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) o ... Err_dz_deriv_af -0.110833 0.147709 0.147094 0.116647 ... 0 1 0 1 ... I 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ I 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... 0 1 drop W @ I -0.760828 0.764471 0.621803 0.638864 -0.002334 0.760815 0.262031 -0.046176 ... Err_dz_deriv_af ( 0 1 drop W @ I ) +------------------------------------+-------------------+ |-0.110833 0.147709 0.147094 0.116647|-0.760828 0.764471| | | 0.621803 0.638864| | |-0.002334 0.760815| | | 0.262031 -0.046176| +------------------------------------+-------------------+ ... INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.206392 0.116163 ... Err_dz := INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.206392 0.116163 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.489241 0.598459 ... rest Outputs @ I 0.489241 0.598459 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------+ |ann_d_log_activ|0.489241 0.598459| +---------------+-----------------+ ...the arguments for the opform are ... Y ... Y 0.489241 0.598459 ... Y 0.489241 0.598459 ... 1 - Y 0.510759 0.401541 ... Y * ( 1 - Y ) 0.249884 0.240306 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.249884 0.240306 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.249884 0.240306 ... Err_dz 0.206392 0.116163 ... Deriv_af 0.249884 0.240306 ... Err_dz * Deriv_af 0.0515742 0.0279146 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.0515742 0.0279146 ... I 1 ... I - 1 0 ... Grad_e +--------------+-------------------------------+ |0. 0. 0. 0. 0.|-0.0810253 -0.054224 -0.0663289| |0. 0. 0. 0. 0.| 0.107984 0.0722653 0.0883978| | | 0.107534 0.0719645 0.0880298| | | 0.0852757 0.0570684 0.0698083| +--------------+-------------------------------+ ... Grad_e @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Err_dz_deriv_af 0.0515742 0.0279146 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.731059 0.5 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.731059 0.5 0.5 0.5 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +-------------------+-----------------------------+ |0.0515742 0.0279146|0.731059 0.731059 0.5 0.5 0.5| +-------------------+-----------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0.0377038 0.0377038 0.0257871 0.0257871 0.0257871 0.0204072 0.0204072 0.0139573 0.0139573 0.0139573 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.0377038 0.0377038 0.0257871 0.0257871 0.0257871 0.0204072 0.0204072 0.0139573 0.0139573 0.0139573 ... I 1 ... I - 1 0 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +-------------------------------------------------+-------------------------------+ |0.0377038 0.0377038 0.0257871 0.0257871 0.0257871|-0.0810253 -0.054224 -0.0663289| |0.0204072 0.0204072 0.0139573 0.0139573 0.0139573| 0.107984 0.0722653 0.0883978| | | 0.107534 0.0719645 0.0880298| | | 0.0852757 0.0570684 0.0698083| +-------------------------------------------------+-------------------------------+ ... Layers 3 ... tell Layers 0 1 2 ... I 0 ... I = 0 l ... P 1 ... [ , P ] ?noexpr 1 ... X 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X | [ , P ] 0 1 0 0 ... Out_base := X | [ , P ] 0 1 0 0 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 0 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0 1 0 0 ... Bias @ I link Out_base 1 0 1 0 0 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+---------+ |ann_log_activ|1 0 1 0 0| +-------------+---------+ ...the arguments for the opform are ... X ... X 1 0 1 0 0 ... -1 * X -1 0 -1 0 0 ... 1 + exp ( -1 * X ) 1.36788 2. 1.36788 2. 2. ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.5 0.731059 0.5 0.5 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.5 0.731059 0.5 0.5 ... I 0 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... I 1 ... I = 0 o ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.489241 0.598459|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.731059 0.5 0.5 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0.731059 0.5 0.731059 0.5 0.5| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+-----------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) -0.0726668 0.18563 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) -0.0726668 0.18563 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base -0.0726668 0.18563 ... Bias @ I link Out_base 1 -0.0726668 0.18563 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+--------------------+ |ann_log_activ|1 -0.0726668 0.18563| +-------------+--------------------+ ...the arguments for the opform are ... X ... X 1 -0.0726668 0.18563 ... -1 * X -1 0.0726668 -0.18563 ... 1 + exp ( -1 * X ) 1.36788 2.07537 1.83058 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.481841 0.546275 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.481841 0.546275 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... I 2 ... I = 0 o ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.551849 0.645487 0.633652 0.468452| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.481841 0.546275 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+--------------------------+ | 0.16807 -0.760828 0.764471|0.731059 0.481841 0.546275| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+--------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.173882 0.561319 0.50823 -0.125888 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.173882 0.561319 0.50823 -0.125888 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0.173882 0.561319 0.50823 -0.125888 ... Bias @ I link Out_base 1 0.173882 0.561319 0.50823 -0.125888 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+-------------------------------------+ |ann_log_activ|1 0.173882 0.561319 0.50823 -0.125888| +-------------+-------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.173882 0.561319 0.50823 -0.125888 ... -1 * X -1 -0.173882 -0.561319 -0.50823 0.125888 ... 1 + exp ( -1 * X ) 1.36788 1.8404 1.57046 1.60156 2.13416 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.543361 0.636758 0.624391 0.468569 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.543361 0.636758 0.624391 0.468569 ... I 2 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Layers 3 ... Layers - 1 2 ... reverse count ( Layers - 1 ) 2 1 ... I 2 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) l ... Err_deriv_y ann_d_sum_of_sqr ... I 2 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.543361 0.636758 0.624391 0.468569 ... rest Outputs @ I 0.543361 0.636758 0.624391 0.468569 ... P 1 ... [ , P ] ?noexpr 1 ... X_t 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X_t | [ , P ] 0 1 0 0 ... ( rest Outputs @ I ) X_t | [ , P ] +-----------------------------------+-------+ |0.543361 0.636758 0.624391 0.468569|0 1 0 0| +-----------------------------------+-------+ ... Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) +----------------+---------------------------------------------+ |ann_d_sum_of_sqr|+-----------------------------------+-------+| | ||0.543361 0.636758 0.624391 0.468569|0 1 0 0|| | |+-----------------------------------+-------+| +----------------+---------------------------------------------+ ...the arguments for the opform are ... Y T ... Y 0.543361 0.636758 0.624391 0.468569 ... T 0 1 0 0 ... Y - T 0.543361 -0.363242 0.624391 0.468569 ...end of operation call ... apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.543361 -0.363242 0.624391 0.468569 ... Err_dz := apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.543361 -0.363242 0.624391 0.468569 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 2 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.543361 0.636758 0.624391 0.468569 ... rest Outputs @ I 0.543361 0.636758 0.624391 0.468569 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------------------------+ |ann_d_log_activ|0.543361 0.636758 0.624391 0.468569| +---------------+-----------------------------------+ ...the arguments for the opform are ... Y ... Y 0.543361 0.636758 0.624391 0.468569 ... Y 0.543361 0.636758 0.624391 0.468569 ... 1 - Y 0.456639 0.363242 0.375609 0.531431 ... Y * ( 1 - Y ) 0.24812 0.231297 0.234527 0.249012 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.24812 0.231297 0.234527 0.249012 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.24812 0.231297 0.234527 0.249012 ... Err_dz 0.543361 -0.363242 0.624391 0.468569 ... Deriv_af 0.24812 0.231297 0.234527 0.249012 ... Err_dz * Deriv_af 0.134819 -0.0840169 0.146437 0.116679 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.134819 -0.0840169 0.146437 0.116679 ... I 2 ... I - 1 1 ... Grad_e +-------------------------------------------------+-------------------------------+ |0.0377038 0.0377038 0.0257871 0.0257871 0.0257871|-0.0810253 -0.054224 -0.0663289| |0.0204072 0.0204072 0.0139573 0.0139573 0.0139573| 0.107984 0.0722653 0.0883978| | | 0.107534 0.0719645 0.0880298| | | 0.0852757 0.0570684 0.0698083| +-------------------------------------------------+-------------------------------+ ... Grad_e @ ( I - 1 ) -0.0810253 -0.054224 -0.0663289 0.107984 0.0722653 0.0883978 0.107534 0.0719645 0.0880298 0.0852757 0.0570684 0.0698083 ... Err_dz_deriv_af 0.134819 -0.0840169 0.146437 0.116679 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.481841 0.546275 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +-------------------------------------+--------------------------+ |0.134819 -0.0840169 0.146437 0.116679|0.731059 0.481841 0.546275| +-------------------------------------+--------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0.0985604 0.0649612 0.0736481 -0.0614213 -0.0404828 -0.0458963 0.107054 0.0705592 0.0799946 0.0852995 0.056221 0.0637391 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.017535 0.0107372 0.00731911 0.0465627 0.0317825 0.0425015 0.214588 0.142524 0.168024 0.170575 0.113289 0.133547 ... I 2 ... I - 1 1 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +-------------------------------------------------+------------------------------+ |0.0377038 0.0377038 0.0257871 0.0257871 0.0257871| 0.017535 0.0107372 0.00731911| |0.0204072 0.0204072 0.0139573 0.0139573 0.0139573|0.0465627 0.0317825 0.0425015| | | 0.214588 0.142524 0.168024| | | 0.170575 0.113289 0.133547| +-------------------------------------------------+------------------------------+ ... I 1 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) o ... Err_dz_deriv_af 0.134819 -0.0840169 0.146437 0.116679 ... 0 1 0 1 ... I 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ I 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... 0 1 drop W @ I -0.760828 0.764471 0.621803 0.638864 -0.002334 0.760815 0.262031 -0.046176 ... Err_dz_deriv_af ( 0 1 drop W @ I ) +-------------------------------------+-------------------+ |0.134819 -0.0840169 0.146437 0.116679|-0.760828 0.764471| | | 0.621803 0.638864| | |-0.002334 0.760815| | | 0.262031 -0.046176| +-------------------------------------+-------------------+ ... INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) -0.124584 0.155413 ... Err_dz := INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) -0.124584 0.155413 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.481841 0.546275 ... rest Outputs @ I 0.481841 0.546275 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------+ |ann_d_log_activ|0.481841 0.546275| +---------------+-----------------+ ...the arguments for the opform are ... Y ... Y 0.481841 0.546275 ... Y 0.481841 0.546275 ... 1 - Y 0.518159 0.453725 ... Y * ( 1 - Y ) 0.24967 0.247859 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.24967 0.247859 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.24967 0.247859 ... Err_dz -0.124584 0.155413 ... Deriv_af 0.24967 0.247859 ... Err_dz * Deriv_af -0.0311049 0.0385204 ... Err_dz_deriv_af := Err_dz * Deriv_af -0.0311049 0.0385204 ... I 1 ... I - 1 0 ... Grad_e +-------------------------------------------------+------------------------------+ |0.0377038 0.0377038 0.0257871 0.0257871 0.0257871| 0.017535 0.0107372 0.00731911| |0.0204072 0.0204072 0.0139573 0.0139573 0.0139573|0.0465627 0.0317825 0.0425015| | | 0.214588 0.142524 0.168024| | | 0.170575 0.113289 0.133547| +-------------------------------------------------+------------------------------+ ... Grad_e @ ( I - 1 ) 0.0377038 0.0377038 0.0257871 0.0257871 0.0257871 0.0204072 0.0204072 0.0139573 0.0139573 0.0139573 ... Err_dz_deriv_af -0.0311049 0.0385204 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.731059 0.5 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.731059 0.5 0.5 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +--------------------+-----------------------------+ |-0.0311049 0.0385204|0.731059 0.5 0.731059 0.5 0.5| +--------------------+-----------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) -0.0227395 -0.0155525 -0.0227395 -0.0155525 -0.0155525 0.0281607 0.0192602 0.0281607 0.0192602 0.0192602 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.0149643 0.0221513 0.0030476 0.0102347 0.0102347 0.0485679 0.0396674 0.042118 0.0332175 0.0332175 ... I 1 ... I - 1 0 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +-------------------------------------------------+------------------------------+ |0.0149643 0.0221513 0.0030476 0.0102347 0.0102347| 0.017535 0.0107372 0.00731911| |0.0485679 0.0396674 0.042118 0.0332175 0.0332175|0.0465627 0.0317825 0.0425015| | | 0.214588 0.142524 0.168024| | | 0.170575 0.113289 0.133547| +-------------------------------------------------+------------------------------+ ... Layers 3 ... tell Layers 0 1 2 ... I 0 ... I = 0 l ... P 2 ... [ , P ] ?noexpr 2 ... X 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X | [ , P ] 0 0 1 0 ... Out_base := X | [ , P ] 0 0 1 0 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 0 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0 0 1 0 ... Bias @ I link Out_base 1 0 0 1 0 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+---------+ |ann_log_activ|1 0 0 1 0| +-------------+---------+ ...the arguments for the opform are ... X ... X 1 0 0 1 0 ... -1 * X -1 0 0 -1 0 ... 1 + exp ( -1 * X ) 1.36788 2. 2. 1.36788 2. ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.5 0.5 0.731059 0.5 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.5 0.5 0.731059 0.5 ... I 0 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... I 1 ... I = 0 o ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.481841 0.546275|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 0.731059 0.5 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0.731059 0.5 0.5 0.731059 0.5| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+-----------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.194072 0.226155 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.194072 0.226155 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0.194072 0.226155 ... Bias @ I link Out_base 1 0.194072 0.226155 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+-------------------+ |ann_log_activ|1 0.194072 0.226155| +-------------+-------------------+ ...the arguments for the opform are ... X ... X 1 0.194072 0.226155 ... -1 * X -1 -0.194072 -0.226155 ... 1 + exp ( -1 * X ) 1.36788 1.8236 1.79759 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.548366 0.556299 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.548366 0.556299 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... I 2 ... I = 0 o ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.543361 0.636758 0.624391 0.468569| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.548366 0.556299 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+--------------------------+ | 0.16807 -0.760828 0.764471|0.731059 0.548366 0.556299| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+--------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.130931 0.609089 0.515701 -0.108919 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.130931 0.609089 0.515701 -0.108919 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0.130931 0.609089 0.515701 -0.108919 ... Bias @ I link Out_base 1 0.130931 0.609089 0.515701 -0.108919 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+--------------------------------------+ |ann_log_activ|1 0.130931 0.609089 0.515701 -0.108919| +-------------+--------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.130931 0.609089 0.515701 -0.108919 ... -1 * X -1 -0.130931 -0.609089 -0.515701 0.108919 ... 1 + exp ( -1 * X ) 1.36788 1.87728 1.54385 1.59708 2.11507 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.532686 0.647733 0.626142 0.472797 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.532686 0.647733 0.626142 0.472797 ... I 2 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... Layers 3 ... Layers - 1 2 ... reverse count ( Layers - 1 ) 2 1 ... I 2 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) l ... Err_deriv_y ann_d_sum_of_sqr ... I 2 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.532686 0.647733 0.626142 0.472797 ... rest Outputs @ I 0.532686 0.647733 0.626142 0.472797 ... P 2 ... [ , P ] ?noexpr 2 ... X_t 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X_t | [ , P ] 0 0 1 0 ... ( rest Outputs @ I ) X_t | [ , P ] +-----------------------------------+-------+ |0.532686 0.647733 0.626142 0.472797|0 0 1 0| +-----------------------------------+-------+ ... Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) +----------------+---------------------------------------------+ |ann_d_sum_of_sqr|+-----------------------------------+-------+| | ||0.532686 0.647733 0.626142 0.472797|0 0 1 0|| | |+-----------------------------------+-------+| +----------------+---------------------------------------------+ ...the arguments for the opform are ... Y T ... Y 0.532686 0.647733 0.626142 0.472797 ... T 0 0 1 0 ... Y - T 0.532686 0.647733 -0.373858 0.472797 ...end of operation call ... apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.532686 0.647733 -0.373858 0.472797 ... Err_dz := apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.532686 0.647733 -0.373858 0.472797 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 2 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.532686 0.647733 0.626142 0.472797 ... rest Outputs @ I 0.532686 0.647733 0.626142 0.472797 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------------------------+ |ann_d_log_activ|0.532686 0.647733 0.626142 0.472797| +---------------+-----------------------------------+ ...the arguments for the opform are ... Y ... Y 0.532686 0.647733 0.626142 0.472797 ... Y 0.532686 0.647733 0.626142 0.472797 ... 1 - Y 0.467314 0.352267 0.373858 0.527203 ... Y * ( 1 - Y ) 0.248932 0.228175 0.234088 0.24926 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.248932 0.228175 0.234088 0.24926 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.248932 0.228175 0.234088 0.24926 ... Err_dz 0.532686 0.647733 -0.373858 0.472797 ... Deriv_af 0.248932 0.228175 0.234088 0.24926 ... Err_dz * Deriv_af 0.132602 0.147796 -0.0875157 0.117849 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.132602 0.147796 -0.0875157 0.117849 ... I 2 ... I - 1 1 ... Grad_e +-------------------------------------------------+------------------------------+ |0.0149643 0.0221513 0.0030476 0.0102347 0.0102347| 0.017535 0.0107372 0.00731911| |0.0485679 0.0396674 0.042118 0.0332175 0.0332175|0.0465627 0.0317825 0.0425015| | | 0.214588 0.142524 0.168024| | | 0.170575 0.113289 0.133547| +-------------------------------------------------+------------------------------+ ... Grad_e @ ( I - 1 ) 0.017535 0.0107372 0.00731911 0.0465627 0.0317825 0.0425015 0.214588 0.142524 0.168024 0.170575 0.113289 0.133547 ... Err_dz_deriv_af 0.132602 0.147796 -0.0875157 0.117849 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.548366 0.556299 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +-------------------------------------+--------------------------+ |0.132602 0.147796 -0.0875157 0.117849|0.731059 0.548366 0.556299| +-------------------------------------+--------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0.0969401 0.0727147 0.0737666 0.108048 0.0810466 0.082219 -0.0639791 -0.0479907 -0.0486849 0.0861548 0.0646246 0.0655595 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.114475 0.0834519 0.0810857 0.154611 0.112829 0.12472 0.150609 0.094533 0.119339 0.25673 0.177914 0.199107 ... I 2 ... I - 1 1 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +-------------------------------------------------+----------------------------+ |0.0149643 0.0221513 0.0030476 0.0102347 0.0102347|0.114475 0.0834519 0.0810857| |0.0485679 0.0396674 0.042118 0.0332175 0.0332175|0.154611 0.112829 0.12472| | |0.150609 0.094533 0.119339| | | 0.25673 0.177914 0.199107| +-------------------------------------------------+----------------------------+ ... I 1 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) o ... Err_dz_deriv_af 0.132602 0.147796 -0.0875157 0.117849 ... 0 1 0 1 ... I 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ I 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... 0 1 drop W @ I -0.760828 0.764471 0.621803 0.638864 -0.002334 0.760815 0.262031 -0.046176 ... Err_dz_deriv_af ( 0 1 drop W @ I ) +-------------------------------------+-------------------+ |0.132602 0.147796 -0.0875157 0.117849|-0.760828 0.764471| | | 0.621803 0.638864| | |-0.002334 0.760815| | | 0.262031 -0.046176| +-------------------------------------+-------------------+ ... INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.0220971 0.123767 ... Err_dz := INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.0220971 0.123767 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 1 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.548366 0.556299 ... rest Outputs @ I 0.548366 0.556299 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------+ |ann_d_log_activ|0.548366 0.556299| +---------------+-----------------+ ...the arguments for the opform are ... Y ... Y 0.548366 0.556299 ... Y 0.548366 0.556299 ... 1 - Y 0.451634 0.443701 ... Y * ( 1 - Y ) 0.247661 0.24683 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.247661 0.24683 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.247661 0.24683 ... Err_dz 0.0220971 0.123767 ... Deriv_af 0.247661 0.24683 ... Err_dz * Deriv_af 0.00547259 0.0305496 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.00547259 0.0305496 ... I 1 ... I - 1 0 ... Grad_e +-------------------------------------------------+----------------------------+ |0.0149643 0.0221513 0.0030476 0.0102347 0.0102347|0.114475 0.0834519 0.0810857| |0.0485679 0.0396674 0.042118 0.0332175 0.0332175|0.154611 0.112829 0.12472| | |0.150609 0.094533 0.119339| | | 0.25673 0.177914 0.199107| +-------------------------------------------------+----------------------------+ ... Grad_e @ ( I - 1 ) 0.0149643 0.0221513 0.0030476 0.0102347 0.0102347 0.0485679 0.0396674 0.042118 0.0332175 0.0332175 ... Err_dz_deriv_af 0.00547259 0.0305496 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.731059 0.5|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 0.731059 0.5 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +--------------------+-----------------------------+ |0.00547259 0.0305496|0.731059 0.5 0.5 0.731059 0.5| +--------------------+-----------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0.00400078 0.00273629 0.00273629 0.00400078 0.00273629 0.0223335 0.0152748 0.0152748 0.0223335 0.0152748 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.0189651 0.0248876 0.0057839 0.0142354 0.012971 0.0709014 0.0549422 0.0573927 0.055551 0.0484923 ... I 1 ... I - 1 0 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +-------------------------------------------------+----------------------------+ |0.0189651 0.0248876 0.0057839 0.0142354 0.012971|0.114475 0.0834519 0.0810857| |0.0709014 0.0549422 0.0573927 0.055551 0.0484923|0.154611 0.112829 0.12472| | |0.150609 0.094533 0.119339| | | 0.25673 0.177914 0.199107| +-------------------------------------------------+----------------------------+ ... Layers 3 ... tell Layers 0 1 2 ... I 0 ... I = 0 l ... P 3 ... [ , P ] ?noexpr 3 ... X 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X | [ , P ] 0 0 0 1 ... Out_base := X | [ , P ] 0 0 0 1 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 0 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0 0 0 1 ... Bias @ I link Out_base 1 0 0 0 1 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+---------+ |ann_log_activ|1 0 0 0 1| +-------------+---------+ ...the arguments for the opform are ... X ... X 1 0 0 0 1 ... -1 * X -1 0 0 0 -1 ... 1 + exp ( -1 * X ) 1.36788 2. 2. 2. 1.36788 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.5 0.5 0.5 0.731059 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.5 0.5 0.5 0.731059 ... I 0 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... I 1 ... I = 0 o ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+--------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.548366 0.556299|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+--------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 0.5 0.731059 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0.731059 0.5 0.5 0.5 0.731059| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+-----------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) -0.193223 0.156158 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) -0.193223 0.156158 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base -0.193223 0.156158 ... Bias @ I link Out_base 1 -0.193223 0.156158 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+--------------------+ |ann_log_activ|1 -0.193223 0.156158| +-------------+--------------------+ ...the arguments for the opform are ... X ... X 1 -0.193223 0.156158 ... -1 * X -1 0.193223 -0.156158 ... 1 + exp ( -1 * X ) 1.36788 2.21315 1.85542 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.451844 0.53896 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.451844 0.53896 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+-------------------------+--------------------------------------------+ ... I 2 ... I = 0 o ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.532686 0.647733 0.626142 0.472797| +-----------------------------+-------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.451844 0.53896 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+-------------------------+ | 0.16807 -0.760828 0.764471|0.731059 0.451844 0.53896| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+-------------------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.191113 0.537994 0.502735 -0.13341 ... Out_base := INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.191113 0.537994 0.502735 -0.13341 ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... Out_base 0.191113 0.537994 0.502735 -0.13341 ... Bias @ I link Out_base 1 0.191113 0.537994 0.502735 -0.13341 ... Af @ 0 ( Bias @ I link Out_base ) +-------------+-------------------------------------+ |ann_log_activ|1 0.191113 0.537994 0.502735 -0.13341| +-------------+-------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.191113 0.537994 0.502735 -0.13341 ... -1 * X -1 -0.191113 -0.537994 -0.502735 0.13341 ... 1 + exp ( -1 * X ) 1.36788 1.82604 1.58392 1.60487 2.14272 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.547633 0.631346 0.623102 0.466697 ...end of operation call ... apply Af @ 0 ( Bias @ I link Out_base ) 0.731059 0.547633 0.631346 0.623102 0.466697 ... I 2 ... Outputs @ I := apply Af @ 0 ( Bias @ I link Out_base ) +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.547633 0.631346 0.623102 0.466697| +-----------------------------+-------------------------+--------------------------------------------+ ... Layers 3 ... Layers - 1 2 ... reverse count ( Layers - 1 ) 2 1 ... I 2 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) l ... Err_deriv_y ann_d_sum_of_sqr ... I 2 ... Outputs +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.547633 0.631346 0.623102 0.466697| +-----------------------------+-------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.547633 0.631346 0.623102 0.466697 ... rest Outputs @ I 0.547633 0.631346 0.623102 0.466697 ... P 3 ... [ , P ] ?noexpr 3 ... X_t 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X_t | [ , P ] 0 0 0 1 ... ( rest Outputs @ I ) X_t | [ , P ] +-----------------------------------+-------+ |0.547633 0.631346 0.623102 0.466697|0 0 0 1| +-----------------------------------+-------+ ... Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) +----------------+---------------------------------------------+ |ann_d_sum_of_sqr|+-----------------------------------+-------+| | ||0.547633 0.631346 0.623102 0.466697|0 0 0 1|| | |+-----------------------------------+-------+| +----------------+---------------------------------------------+ ...the arguments for the opform are ... Y T ... Y 0.547633 0.631346 0.623102 0.466697 ... T 0 0 0 1 ... Y - T 0.547633 0.631346 0.623102 -0.533303 ...end of operation call ... apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.547633 0.631346 0.623102 -0.533303 ... Err_dz := apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.547633 0.631346 0.623102 -0.533303 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 2 ... Outputs +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.547633 0.631346 0.623102 0.466697| +-----------------------------+-------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.547633 0.631346 0.623102 0.466697 ... rest Outputs @ I 0.547633 0.631346 0.623102 0.466697 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------------------------+ |ann_d_log_activ|0.547633 0.631346 0.623102 0.466697| +---------------+-----------------------------------+ ...the arguments for the opform are ... Y ... Y 0.547633 0.631346 0.623102 0.466697 ... Y 0.547633 0.631346 0.623102 0.466697 ... 1 - Y 0.452367 0.368654 0.376898 0.533303 ... Y * ( 1 - Y ) 0.247731 0.232748 0.234846 0.248891 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.247731 0.232748 0.234846 0.248891 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.247731 0.232748 0.234846 0.248891 ... Err_dz 0.547633 0.631346 0.623102 -0.533303 ... Deriv_af 0.247731 0.232748 0.234846 0.248891 ... Err_dz * Deriv_af 0.135666 0.146945 0.146333 -0.132734 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.135666 0.146945 0.146333 -0.132734 ... I 2 ... I - 1 1 ... Grad_e +-------------------------------------------------+----------------------------+ |0.0189651 0.0248876 0.0057839 0.0142354 0.012971|0.114475 0.0834519 0.0810857| |0.0709014 0.0549422 0.0573927 0.055551 0.0484923|0.154611 0.112829 0.12472| | |0.150609 0.094533 0.119339| | | 0.25673 0.177914 0.199107| +-------------------------------------------------+----------------------------+ ... Grad_e @ ( I - 1 ) 0.114475 0.0834519 0.0810857 0.154611 0.112829 0.12472 0.150609 0.094533 0.119339 0.25673 0.177914 0.199107 ... Err_dz_deriv_af 0.135666 0.146945 0.146333 -0.132734 ... I 2 ... I - 1 1 ... Outputs +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.547633 0.631346 0.623102 0.466697| +-----------------------------+-------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.451844 0.53896 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +------------------------------------+-------------------------+ |0.135666 0.146945 0.146333 -0.132734|0.731059 0.451844 0.53896| +------------------------------------+-------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0.0991796 0.0612998 0.0731185 0.107425 0.0663961 0.0791973 0.106978 0.0661197 0.0788677 -0.0970366 -0.0599752 -0.0715385 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.213655 0.144752 0.154204 0.262036 0.179225 0.203918 0.257587 0.160653 0.198207 0.159693 0.117939 0.127568 ... I 2 ... I - 1 1 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +-------------------------------------------------+--------------------------+ |0.0189651 0.0248876 0.0057839 0.0142354 0.012971|0.213655 0.144752 0.154204| |0.0709014 0.0549422 0.0573927 0.055551 0.0484923|0.262036 0.179225 0.203918| | |0.257587 0.160653 0.198207| | |0.159693 0.117939 0.127568| +-------------------------------------------------+--------------------------+ ... I 1 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) o ... Err_dz_deriv_af 0.135666 0.146945 0.146333 -0.132734 ... 0 1 0 1 ... I 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ I 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... 0 1 drop W @ I -0.760828 0.764471 0.621803 0.638864 -0.002334 0.760815 0.262031 -0.046176 ... Err_dz_deriv_af ( 0 1 drop W @ I ) +------------------------------------+-------------------+ |0.135666 0.146945 0.146333 -0.132734|-0.760828 0.764471| | | 0.621803 0.638864| | |-0.002334 0.760815| | | 0.262031 -0.046176| +------------------------------------+-------------------+ ... INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) -0.0469698 0.315052 ... Err_dz := INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) -0.0469698 0.315052 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 1 ... Outputs +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.547633 0.631346 0.623102 0.466697| +-----------------------------+-------------------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.451844 0.53896 ... rest Outputs @ I 0.451844 0.53896 ... Af @ 1 ( rest Outputs @ I ) +---------------+----------------+ |ann_d_log_activ|0.451844 0.53896| +---------------+----------------+ ...the arguments for the opform are ... Y ... Y 0.451844 0.53896 ... Y 0.451844 0.53896 ... 1 - Y 0.548156 0.46104 ... Y * ( 1 - Y ) 0.247681 0.248482 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.247681 0.248482 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.247681 0.248482 ... Err_dz -0.0469698 0.315052 ... Deriv_af 0.247681 0.248482 ... Err_dz * Deriv_af -0.0116335 0.0782847 ... Err_dz_deriv_af := Err_dz * Deriv_af -0.0116335 0.0782847 ... I 1 ... I - 1 0 ... Grad_e +-------------------------------------------------+--------------------------+ |0.0189651 0.0248876 0.0057839 0.0142354 0.012971|0.213655 0.144752 0.154204| |0.0709014 0.0549422 0.0573927 0.055551 0.0484923|0.262036 0.179225 0.203918| | |0.257587 0.160653 0.198207| | |0.159693 0.117939 0.127568| +-------------------------------------------------+--------------------------+ ... Grad_e @ ( I - 1 ) 0.0189651 0.0248876 0.0057839 0.0142354 0.012971 0.0709014 0.0549422 0.0573927 0.055551 0.0484923 ... Err_dz_deriv_af -0.0116335 0.0782847 ... I 1 ... I - 1 0 ... Outputs +-----------------------------+-------------------------+--------------------------------------------+ |0.731059 0.5 0.5 0.5 0.731059|0.731059 0.451844 0.53896|0.731059 0.547633 0.631346 0.623102 0.466697| +-----------------------------+-------------------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 0.5 0.731059 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +--------------------+-----------------------------+ |-0.0116335 0.0782847|0.731059 0.5 0.5 0.5 0.731059| +--------------------+-----------------------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) -0.00850478 -0.00581676 -0.00581676 -0.00581676 -0.00850478 0.0572307 0.0391423 0.0391423 0.0391423 0.0572307 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.0104603 0.0190709 -3.28639e-05 0.00841868 0.00446617 0.128132 0.0940845 0.0965351 0.0946934 0.105723 ... I 1 ... I - 1 0 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +------------------------------------------------------+--------------------------+ |0.0104603 0.0190709 -3.28639e-05 0.00841868 0.00446617|0.213655 0.144752 0.154204| | 0.128132 0.0940845 0.0965351 0.0946934 0.105723|0.262036 0.179225 0.203918| | |0.257587 0.160653 0.198207| | |0.159693 0.117939 0.127568| +------------------------------------------------------+--------------------------+ ... Grad_e +------------------------------------------------------+--------------------------+ |0.0104603 0.0190709 -3.28639e-05 0.00841868 0.00446617|0.213655 0.144752 0.154204| | 0.128132 0.0940845 0.0965351 0.0946934 0.105723|0.262036 0.179225 0.203918| | |0.257587 0.160653 0.198207| | |0.159693 0.117939 0.127568| +------------------------------------------------------+--------------------------+ ...end of operation call +------------------------------------------------------+--------------------------+ |0.0104603 0.0190709 -3.28639e-05 0.00841868 0.00446617|0.213655 0.144752 0.154204| | 0.128132 0.0940845 0.0965351 0.0946934 0.105723|0.262036 0.179225 0.203918| | |0.257587 0.160653 0.198207| | |0.159693 0.117939 0.127568| +------------------------------------------------------+--------------------------+ set "nolog nolog ann_FF_grad_BP x x_t N W c af err_deriv_y ...trace call to operation ... ann_ff_grad_bp ...the arguments for the opform are ... X X_t N W C Af Err_deriv_y ... N 4 2 4 ... gage shape N 3 ... Layers := gage shape N 3 ... X 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... gage second shape X 4 ... Patterns := gage second shape X 4 ... N 4 2 4 ... 1 N +-+-----+ |1|4 2 4| +-+-----+ ... + 1 N 5 3 5 ... N_bias := + 1 N 5 3 5 ... N_bias 5 3 5 ... N_bias EACHLEFT reshape 0 +---------+-----+---------+ |0 0 0 0 0|0 0 0|0 0 0 0 0| +---------+-----+---------+ ... Outputs := N_bias EACHLEFT reshape 0 +---------+-----+---------+ |0 0 0 0 0|0 0 0|0 0 0 0 0| +---------+-----+---------+ ... Err_dz_deriv_af := 0 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... 0 * W +--------------+--------+ |0. 0. 0. 0. 0.|0. 0. 0.| |0. 0. 0. 0. 0.|0. 0. 0.| | |0. 0. 0.| | |0. 0. 0.| +--------------+--------+ ... Grad_e := 0 * W +--------------+--------+ |0. 0. 0. 0. 0.|0. 0. 0.| |0. 0. 0. 0. 0.|0. 0. 0.| | |0. 0. 0.| | |0. 0. 0.| +--------------+--------+ ... Patterns 4 ... tell Patterns 0 1 2 3 ... Layers 3 ... Layers - 1 2 ... count ( Layers - 1 ) 1 2 ... I 1 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +---------+-----+---------+ |0 0 0 0 0|0 0 0|0 0 0 0 0| +---------+-----+---------+ ... Outputs @ ( I - 1 ) 0 0 0 0 0 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+---------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0 0 0 0 0| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+---------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0. 0. ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0. 0. ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+-------+ |ann_log_activ|1 0. 0.| +-------------+-------+ ...the arguments for the opform are ... X ... X 1 0. 0. ... -1 * X -1 0. 0. ... 1 + exp ( -1 * X ) 1.36788 2. 2. ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.5 0.5 ...end of operation call ... apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) 0.731059 0.5 0.5 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +---------+----------------+---------+ |0 0 0 0 0|0.731059 0.5 0.5|0 0 0 0 0| +---------+----------------+---------+ ... I 2 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +---------+----------------+---------+ |0 0 0 0 0|0.731059 0.5 0.5|0 0 0 0 0| +---------+----------------+---------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+----------------+ | 0.16807 -0.760828 0.764471|0.731059 0.5 0.5| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+----------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.124691 0.543047 0.472981 -0.118993 ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0.124691 0.543047 0.472981 -0.118993 ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+--------------------------------------+ |ann_log_activ|1 0.124691 0.543047 0.472981 -0.118993| +-------------+--------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.124691 0.543047 0.472981 -0.118993 ... -1 * X -1 -0.124691 -0.543047 -0.472981 0.118993 ... 1 + exp ( -1 * X ) 1.36788 1.88277 1.58098 1.62314 2.12636 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.531132 0.632521 0.616089 0.470287 ...end of operation call ... apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) 0.731059 0.531132 0.632521 0.616089 0.470287 ... I 2 ... Outputs @ I := apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Layers 3 ... Layers - 1 2 ... reverse count ( Layers - 1 ) 2 1 ... I 2 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) l ... Err_deriv_y ann_d_sum_of_sqr ... I 2 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.531132 0.632521 0.616089 0.470287 ... rest Outputs @ I 0.531132 0.632521 0.616089 0.470287 ... P 0 ... [ , P ] ?noexpr 0 ... X_t 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X_t | [ , P ] 1 0 0 0 ... ( rest Outputs @ I ) X_t | [ , P ] +-----------------------------------+-------+ |0.531132 0.632521 0.616089 0.470287|1 0 0 0| +-----------------------------------+-------+ ... Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) +----------------+---------------------------------------------+ |ann_d_sum_of_sqr|+-----------------------------------+-------+| | ||0.531132 0.632521 0.616089 0.470287|1 0 0 0|| | |+-----------------------------------+-------+| +----------------+---------------------------------------------+ ...the arguments for the opform are ... Y T ... Y 0.531132 0.632521 0.616089 0.470287 ... T 1 0 0 0 ... Y - T -0.468868 0.632521 0.616089 0.470287 ...end of operation call ... apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) -0.468868 0.632521 0.616089 0.470287 ... Err_dz := apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) -0.468868 0.632521 0.616089 0.470287 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 2 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.531132 0.632521 0.616089 0.470287 ... rest Outputs @ I 0.531132 0.632521 0.616089 0.470287 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------------------------+ |ann_d_log_activ|0.531132 0.632521 0.616089 0.470287| +---------------+-----------------------------------+ ...the arguments for the opform are ... Y ... Y 0.531132 0.632521 0.616089 0.470287 ... Y 0.531132 0.632521 0.616089 0.470287 ... 1 - Y 0.468868 0.367479 0.383911 0.529713 ... Y * ( 1 - Y ) 0.249031 0.232438 0.236523 0.249117 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.249031 0.232438 0.236523 0.249117 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.249031 0.232438 0.236523 0.249117 ... Err_dz -0.468868 0.632521 0.616089 0.470287 ... Deriv_af 0.249031 0.232438 0.236523 0.249117 ... Err_dz * Deriv_af -0.116762 0.147022 0.145719 0.117156 ... Err_dz_deriv_af := Err_dz * Deriv_af -0.116762 0.147022 0.145719 0.117156 ... I 2 ... I - 1 1 ... Grad_e +--------------+--------+ |0. 0. 0. 0. 0.|0. 0. 0.| |0. 0. 0. 0. 0.|0. 0. 0.| | |0. 0. 0.| | |0. 0. 0.| +--------------+--------+ ... Grad_e @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Err_dz_deriv_af -0.116762 0.147022 0.145719 0.117156 ... I 2 ... I - 1 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +------------------------------------+----------------+ |-0.116762 0.147022 0.145719 0.117156|0.731059 0.5 0.5| +------------------------------------+----------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) -0.0853602 -0.0583812 -0.0583812 0.107482 0.073511 0.073511 0.106529 0.0728597 0.0728597 0.0856483 0.0585782 0.0585782 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) -0.0853602 -0.0583812 -0.0583812 0.107482 0.073511 0.073511 0.106529 0.0728597 0.0728597 0.0856483 0.0585782 0.0585782 ... I 2 ... I - 1 1 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +--------------+--------------------------------+ |0. 0. 0. 0. 0.|-0.0853602 -0.0583812 -0.0583812| |0. 0. 0. 0. 0.| 0.107482 0.073511 0.073511| | | 0.106529 0.0728597 0.0728597| | | 0.0856483 0.0585782 0.0585782| +--------------+--------------------------------+ ... I 1 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) o ... Err_dz_deriv_af -0.116762 0.147022 0.145719 0.117156 ... 0 1 0 1 ... I 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ I 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... 0 1 drop W @ I -0.760828 0.764471 0.621803 0.638864 -0.002334 0.760815 0.262031 -0.046176 ... Err_dz_deriv_af ( 0 1 drop W @ I ) +------------------------------------+-------------------+ |-0.116762 0.147022 0.145719 0.117156|-0.760828 0.764471| | | 0.621803 0.638864| | |-0.002334 0.760815| | | 0.262031 -0.046176| +------------------------------------+-------------------+ ... INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.210613 0.110121 ... Err_dz := INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.210613 0.110121 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.5 0.5 ... rest Outputs @ I 0.5 0.5 ... Af @ 1 ( rest Outputs @ I ) +---------------+-------+ |ann_d_log_activ|0.5 0.5| +---------------+-------+ ...the arguments for the opform are ... Y ... Y 0.5 0.5 ... Y 0.5 0.5 ... 1 - Y 0.5 0.5 ... Y * ( 1 - Y ) 0.25 0.25 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.25 0.25 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.25 0.25 ... Err_dz 0.210613 0.110121 ... Deriv_af 0.25 0.25 ... Err_dz * Deriv_af 0.0526534 0.0275303 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.0526534 0.0275303 ... I 1 ... I - 1 0 ... Grad_e +--------------+--------------------------------+ |0. 0. 0. 0. 0.|-0.0853602 -0.0583812 -0.0583812| |0. 0. 0. 0. 0.| 0.107482 0.073511 0.073511| | | 0.106529 0.0728597 0.0728597| | | 0.0856483 0.0585782 0.0585782| +--------------+--------------------------------+ ... Grad_e @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Err_dz_deriv_af 0.0526534 0.0275303 ... I 1 ... I - 1 0 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0 0 0 0 0 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +-------------------+---------+ |0.0526534 0.0275303|0 0 0 0 0| +-------------------+---------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... I 1 ... I - 1 0 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +--------------+--------------------------------+ |0. 0. 0. 0. 0.|-0.0853602 -0.0583812 -0.0583812| |0. 0. 0. 0. 0.| 0.107482 0.073511 0.073511| | | 0.106529 0.0728597 0.0728597| | | 0.0856483 0.0585782 0.0585782| +--------------+--------------------------------+ ... Layers 3 ... Layers - 1 2 ... count ( Layers - 1 ) 1 2 ... I 1 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0 0 0 0 0 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+---------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0 0 0 0 0| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+---------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0. 0. ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0. 0. ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+-------+ |ann_log_activ|1 0. 0.| +-------------+-------+ ...the arguments for the opform are ... X ... X 1 0. 0. ... -1 * X -1 0. 0. ... 1 + exp ( -1 * X ) 1.36788 2. 2. ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.5 0.5 ...end of operation call ... apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) 0.731059 0.5 0.5 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... I 2 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+----------------+ | 0.16807 -0.760828 0.764471|0.731059 0.5 0.5| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+----------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.124691 0.543047 0.472981 -0.118993 ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0.124691 0.543047 0.472981 -0.118993 ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+--------------------------------------+ |ann_log_activ|1 0.124691 0.543047 0.472981 -0.118993| +-------------+--------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.124691 0.543047 0.472981 -0.118993 ... -1 * X -1 -0.124691 -0.543047 -0.472981 0.118993 ... 1 + exp ( -1 * X ) 1.36788 1.88277 1.58098 1.62314 2.12636 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.531132 0.632521 0.616089 0.470287 ...end of operation call ... apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) 0.731059 0.531132 0.632521 0.616089 0.470287 ... I 2 ... Outputs @ I := apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Layers 3 ... Layers - 1 2 ... reverse count ( Layers - 1 ) 2 1 ... I 2 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) l ... Err_deriv_y ann_d_sum_of_sqr ... I 2 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.531132 0.632521 0.616089 0.470287 ... rest Outputs @ I 0.531132 0.632521 0.616089 0.470287 ... P 1 ... [ , P ] ?noexpr 1 ... X_t 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X_t | [ , P ] 0 1 0 0 ... ( rest Outputs @ I ) X_t | [ , P ] +-----------------------------------+-------+ |0.531132 0.632521 0.616089 0.470287|0 1 0 0| +-----------------------------------+-------+ ... Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) +----------------+---------------------------------------------+ |ann_d_sum_of_sqr|+-----------------------------------+-------+| | ||0.531132 0.632521 0.616089 0.470287|0 1 0 0|| | |+-----------------------------------+-------+| +----------------+---------------------------------------------+ ...the arguments for the opform are ... Y T ... Y 0.531132 0.632521 0.616089 0.470287 ... T 0 1 0 0 ... Y - T 0.531132 -0.367479 0.616089 0.470287 ...end of operation call ... apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.531132 -0.367479 0.616089 0.470287 ... Err_dz := apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.531132 -0.367479 0.616089 0.470287 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 2 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.531132 0.632521 0.616089 0.470287 ... rest Outputs @ I 0.531132 0.632521 0.616089 0.470287 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------------------------+ |ann_d_log_activ|0.531132 0.632521 0.616089 0.470287| +---------------+-----------------------------------+ ...the arguments for the opform are ... Y ... Y 0.531132 0.632521 0.616089 0.470287 ... Y 0.531132 0.632521 0.616089 0.470287 ... 1 - Y 0.468868 0.367479 0.383911 0.529713 ... Y * ( 1 - Y ) 0.249031 0.232438 0.236523 0.249117 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.249031 0.232438 0.236523 0.249117 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.249031 0.232438 0.236523 0.249117 ... Err_dz 0.531132 -0.367479 0.616089 0.470287 ... Deriv_af 0.249031 0.232438 0.236523 0.249117 ... Err_dz * Deriv_af 0.132268 -0.0854161 0.145719 0.117156 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.132268 -0.0854161 0.145719 0.117156 ... I 2 ... I - 1 1 ... Grad_e +--------------+--------------------------------+ |0. 0. 0. 0. 0.|-0.0853602 -0.0583812 -0.0583812| |0. 0. 0. 0. 0.| 0.107482 0.073511 0.073511| | | 0.106529 0.0728597 0.0728597| | | 0.0856483 0.0585782 0.0585782| +--------------+--------------------------------+ ... Grad_e @ ( I - 1 ) -0.0853602 -0.0583812 -0.0583812 0.107482 0.073511 0.073511 0.106529 0.0728597 0.0728597 0.0856483 0.0585782 0.0585782 ... Err_dz_deriv_af 0.132268 -0.0854161 0.145719 0.117156 ... I 2 ... I - 1 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +-------------------------------------+----------------+ |0.132268 -0.0854161 0.145719 0.117156|0.731059 0.5 0.5| +-------------------------------------+----------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0.0966959 0.0661341 0.0661341 -0.0624442 -0.0427081 -0.0427081 0.106529 0.0728597 0.0728597 0.0856483 0.0585782 0.0585782 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.0113357 0.0077529 0.0077529 0.0450375 0.0308029 0.0308029 0.213059 0.145719 0.145719 0.171297 0.117156 0.117156 ... I 2 ... I - 1 1 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +--------------+-----------------------------+ |0. 0. 0. 0. 0.|0.0113357 0.0077529 0.0077529| |0. 0. 0. 0. 0.|0.0450375 0.0308029 0.0308029| | | 0.213059 0.145719 0.145719| | | 0.171297 0.117156 0.117156| +--------------+-----------------------------+ ... I 1 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) o ... Err_dz_deriv_af 0.132268 -0.0854161 0.145719 0.117156 ... 0 1 0 1 ... I 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ I 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... 0 1 drop W @ I -0.760828 0.764471 0.621803 0.638864 -0.002334 0.760815 0.262031 -0.046176 ... Err_dz_deriv_af ( 0 1 drop W @ I ) +-------------------------------------+-------------------+ |0.132268 -0.0854161 0.145719 0.117156|-0.760828 0.764471| | | 0.621803 0.638864| | |-0.002334 0.760815| | | 0.262031 -0.046176| +-------------------------------------+-------------------+ ... INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) -0.123387 0.152002 ... Err_dz := INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) -0.123387 0.152002 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.5 0.5 ... rest Outputs @ I 0.5 0.5 ... Af @ 1 ( rest Outputs @ I ) +---------------+-------+ |ann_d_log_activ|0.5 0.5| +---------------+-------+ ...the arguments for the opform are ... Y ... Y 0.5 0.5 ... Y 0.5 0.5 ... 1 - Y 0.5 0.5 ... Y * ( 1 - Y ) 0.25 0.25 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.25 0.25 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.25 0.25 ... Err_dz -0.123387 0.152002 ... Deriv_af 0.25 0.25 ... Err_dz * Deriv_af -0.0308467 0.0380004 ... Err_dz_deriv_af := Err_dz * Deriv_af -0.0308467 0.0380004 ... I 1 ... I - 1 0 ... Grad_e +--------------+-----------------------------+ |0. 0. 0. 0. 0.|0.0113357 0.0077529 0.0077529| |0. 0. 0. 0. 0.|0.0450375 0.0308029 0.0308029| | | 0.213059 0.145719 0.145719| | | 0.171297 0.117156 0.117156| +--------------+-----------------------------+ ... Grad_e @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Err_dz_deriv_af -0.0308467 0.0380004 ... I 1 ... I - 1 0 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0 0 0 0 0 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +--------------------+---------+ |-0.0308467 0.0380004|0 0 0 0 0| +--------------------+---------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... I 1 ... I - 1 0 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +--------------+-----------------------------+ |0. 0. 0. 0. 0.|0.0113357 0.0077529 0.0077529| |0. 0. 0. 0. 0.|0.0450375 0.0308029 0.0308029| | | 0.213059 0.145719 0.145719| | | 0.171297 0.117156 0.117156| +--------------+-----------------------------+ ... Layers 3 ... Layers - 1 2 ... count ( Layers - 1 ) 1 2 ... I 1 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0 0 0 0 0 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+---------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0 0 0 0 0| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+---------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0. 0. ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0. 0. ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+-------+ |ann_log_activ|1 0. 0.| +-------------+-------+ ...the arguments for the opform are ... X ... X 1 0. 0. ... -1 * X -1 0. 0. ... 1 + exp ( -1 * X ) 1.36788 2. 2. ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.5 0.5 ...end of operation call ... apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) 0.731059 0.5 0.5 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... I 2 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+----------------+ | 0.16807 -0.760828 0.764471|0.731059 0.5 0.5| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+----------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.124691 0.543047 0.472981 -0.118993 ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0.124691 0.543047 0.472981 -0.118993 ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+--------------------------------------+ |ann_log_activ|1 0.124691 0.543047 0.472981 -0.118993| +-------------+--------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.124691 0.543047 0.472981 -0.118993 ... -1 * X -1 -0.124691 -0.543047 -0.472981 0.118993 ... 1 + exp ( -1 * X ) 1.36788 1.88277 1.58098 1.62314 2.12636 ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.531132 0.632521 0.616089 0.470287 ...end of operation call ... apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) 0.731059 0.531132 0.632521 0.616089 0.470287 ... I 2 ... Outputs @ I := apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Layers 3 ... Layers - 1 2 ... reverse count ( Layers - 1 ) 2 1 ... I 2 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) l ... Err_deriv_y ann_d_sum_of_sqr ... I 2 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.531132 0.632521 0.616089 0.470287 ... rest Outputs @ I 0.531132 0.632521 0.616089 0.470287 ... P 2 ... [ , P ] ?noexpr 2 ... X_t 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 ... X_t | [ , P ] 0 0 1 0 ... ( rest Outputs @ I ) X_t | [ , P ] +-----------------------------------+-------+ |0.531132 0.632521 0.616089 0.470287|0 0 1 0| +-----------------------------------+-------+ ... Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) +----------------+---------------------------------------------+ |ann_d_sum_of_sqr|+-----------------------------------+-------+| | ||0.531132 0.632521 0.616089 0.470287|0 0 1 0|| | |+-----------------------------------+-------+| +----------------+---------------------------------------------+ ...the arguments for the opform are ... Y T ... Y 0.531132 0.632521 0.616089 0.470287 ... T 0 0 1 0 ... Y - T 0.531132 0.632521 -0.383911 0.470287 ...end of operation call ... apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.531132 0.632521 -0.383911 0.470287 ... Err_dz := apply Err_deriv_y ( ( rest Outputs @ I ) X_t | [ , P ] ) 0.531132 0.632521 -0.383911 0.470287 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 2 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.531132 0.632521 0.616089 0.470287 ... rest Outputs @ I 0.531132 0.632521 0.616089 0.470287 ... Af @ 1 ( rest Outputs @ I ) +---------------+-----------------------------------+ |ann_d_log_activ|0.531132 0.632521 0.616089 0.470287| +---------------+-----------------------------------+ ...the arguments for the opform are ... Y ... Y 0.531132 0.632521 0.616089 0.470287 ... Y 0.531132 0.632521 0.616089 0.470287 ... 1 - Y 0.468868 0.367479 0.383911 0.529713 ... Y * ( 1 - Y ) 0.249031 0.232438 0.236523 0.249117 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.249031 0.232438 0.236523 0.249117 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.249031 0.232438 0.236523 0.249117 ... Err_dz 0.531132 0.632521 -0.383911 0.470287 ... Deriv_af 0.249031 0.232438 0.236523 0.249117 ... Err_dz * Deriv_af 0.132268 0.147022 -0.0908039 0.117156 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.132268 0.147022 -0.0908039 0.117156 ... I 2 ... I - 1 1 ... Grad_e +--------------+-----------------------------+ |0. 0. 0. 0. 0.|0.0113357 0.0077529 0.0077529| |0. 0. 0. 0. 0.|0.0450375 0.0308029 0.0308029| | | 0.213059 0.145719 0.145719| | | 0.171297 0.117156 0.117156| +--------------+-----------------------------+ ... Grad_e @ ( I - 1 ) 0.0113357 0.0077529 0.0077529 0.0450375 0.0308029 0.0308029 0.213059 0.145719 0.145719 0.171297 0.117156 0.117156 ... Err_dz_deriv_af 0.132268 0.147022 -0.0908039 0.117156 ... I 2 ... I - 1 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +-------------------------------------+----------------+ |0.132268 0.147022 -0.0908039 0.117156|0.731059 0.5 0.5| +-------------------------------------+----------------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0.0966959 0.0661341 0.0661341 0.107482 0.073511 0.073511 -0.0663829 -0.0454019 -0.0454019 0.0856483 0.0585782 0.0585782 ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0.108032 0.073887 0.073887 0.152519 0.104314 0.104314 0.146676 0.100318 0.100318 0.256945 0.175735 0.175735 ... I 2 ... I - 1 1 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +--------------+--------------------------+ |0. 0. 0. 0. 0.|0.108032 0.073887 0.073887| |0. 0. 0. 0. 0.|0.152519 0.104314 0.104314| | |0.146676 0.100318 0.100318| | |0.256945 0.175735 0.175735| +--------------+--------------------------+ ... I 1 ... Layers 3 ... Layers - 1 2 ... I = ( Layers - 1 ) o ... Err_dz_deriv_af 0.132268 0.147022 -0.0908039 0.117156 ... 0 1 0 1 ... I 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ I 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... 0 1 drop W @ I -0.760828 0.764471 0.621803 0.638864 -0.002334 0.760815 0.262031 -0.046176 ... Err_dz_deriv_af ( 0 1 drop W @ I ) +-------------------------------------+-------------------+ |0.132268 0.147022 -0.0908039 0.117156|-0.760828 0.764471| | | 0.621803 0.638864| | |-0.002334 0.760815| | | 0.262031 -0.046176| +-------------------------------------+-------------------+ ... INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.0216959 0.120548 ... Err_dz := INNER [ + , * ] Err_dz_deriv_af ( 0 1 drop W @ I ) 0.0216959 0.120548 ... Af ann_log_activ ann_d_log_activ ... Af @ 1 ann_d_log_activ ... I 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ I 0.731059 0.5 0.5 ... rest Outputs @ I 0.5 0.5 ... Af @ 1 ( rest Outputs @ I ) +---------------+-------+ |ann_d_log_activ|0.5 0.5| +---------------+-------+ ...the arguments for the opform are ... Y ... Y 0.5 0.5 ... Y 0.5 0.5 ... 1 - Y 0.5 0.5 ... Y * ( 1 - Y ) 0.25 0.25 ...end of operation call ... apply Af @ 1 ( rest Outputs @ I ) 0.25 0.25 ... Deriv_af := apply Af @ 1 ( rest Outputs @ I ) 0.25 0.25 ... Err_dz 0.0216959 0.120548 ... Deriv_af 0.25 0.25 ... Err_dz * Deriv_af 0.00542397 0.0301369 ... Err_dz_deriv_af := Err_dz * Deriv_af 0.00542397 0.0301369 ... I 1 ... I - 1 0 ... Grad_e +--------------+--------------------------+ |0. 0. 0. 0. 0.|0.108032 0.073887 0.073887| |0. 0. 0. 0. 0.|0.152519 0.104314 0.104314| | |0.146676 0.100318 0.100318| | |0.256945 0.175735 0.175735| +--------------+--------------------------+ ... Grad_e @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Err_dz_deriv_af 0.00542397 0.0301369 ... I 1 ... I - 1 0 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0 0 0 0 0 ... Err_dz_deriv_af Outputs @ ( I - 1 ) +--------------------+---------+ |0.00542397 0.0301369|0 0 0 0 0| +--------------------+---------+ ... OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ... I 1 ... I - 1 0 ... Grad_e @ ( I - 1 ) := Grad_e @ ( I - 1 ) + ( OUTER * Err_dz_deriv_af Outputs @ ( I - 1 ) ) +--------------+--------------------------+ |0. 0. 0. 0. 0.|0.108032 0.073887 0.073887| |0. 0. 0. 0. 0.|0.152519 0.104314 0.104314| | |0.146676 0.100318 0.100318| | |0.256945 0.175735 0.175735| +--------------+--------------------------+ ... Layers 3 ... Layers - 1 2 ... count ( Layers - 1 ) 1 2 ... I 1 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 1 ... Bias 1 1 1 ... Bias @ I 1 ... I 1 ... I - 1 0 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.08784 -0.103661 -0.231872 0.922547 -0.753628 -0.10039 0.822032 -0.10162 0.0737651 -0.229174 ... I 1 ... I - 1 0 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0 0 0 0 0 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +------------------------------------------------+---------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628|0 0 0 0 0| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174| | +------------------------------------------------+---------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0. 0. ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0. 0. ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+-------+ |ann_log_activ|1 0. 0.| +-------------+-------+ ...the arguments for the opform are ... X ... X 1 0. 0. ... -1 * X -1 0. 0. ... 1 + exp ( -1 * X ) 1.36788 2. 2. ... 1 / ( 1 + exp ( -1 * X ) ) 0.731059 0.5 0.5 ...end of operation call ... apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) 0.731059 0.5 0.5 ... I 1 ... Outputs @ I := apply Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... I 2 ... I = 0 o ... Af ann_log_activ ann_d_log_activ ... Af @ 0 ann_log_activ ... I 2 ... Bias 1 1 1 ... Bias @ I 1 ... I 2 ... I - 1 1 ... W +------------------------------------------------+-----------------------------+ | 0.08784 -0.103661 -0.231872 0.922547 -0.753628| 0.16807 -0.760828 0.764471| |-0.10039 0.822032 -0.10162 0.0737651 -0.229174|-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| +------------------------------------------------+-----------------------------+ ... W @ ( I - 1 ) 0.16807 -0.760828 0.764471 -0.119397 0.621803 0.638864 0.128226 -0.002334 0.760815 -0.3104 0.262031 -0.046176 ... I 2 ... I - 1 1 ... Outputs +---------+----------------+--------------------------------------------+ |0 0 0 0 0|0.731059 0.5 0.5|0.731059 0.531132 0.632521 0.616089 0.470287| +---------+----------------+--------------------------------------------+ ... Outputs @ ( I - 1 ) 0.731059 0.5 0.5 ... W @ ( I - 1 ) Outputs @ ( I - 1 ) +-----------------------------+----------------+ | 0.16807 -0.760828 0.764471|0.731059 0.5 0.5| |-0.119397 0.621803 0.638864| | | 0.128226 -0.002334 0.760815| | | -0.3104 0.262031 -0.046176| | +-----------------------------+----------------+ ... INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) 0.124691 0.543047 0.472981 -0.118993 ... Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) 1 0.124691 0.543047 0.472981 -0.118993 ... Af @ 0 ( Bias @ I link ( INNER [ + , * ] W @ ( I - 1 ) Outputs @ ( I - 1 ) ) ) +-------------+--------------------------------------+ |ann_log_activ|1 0.124691 0.543047 0.472981 -0.118993| +-------------+--------------------------------------+ ...the arguments for the opform are ... X ... X 1 0.124691 0.543047 0.472981 -0.118993 ... -1 * X -1 -0.124691 -0.543047 -0.472981 0.118993 Warning: User Break set "nolog