#] #] ********************* #] "$d_Qndfs""code development/strings- develop.txt" # www.BillHowell.ca 10Apr2018 intermediate stage, big problems with quotes # view this file in a text editor, with [constant width font, tab = 3 spaces], no line-wrap My basic problem was a lack of understanding how bash processes strings with apostophes and quotes BUT also - use sed with 's###' rather than 's///' +-----+ ToDos : 12Jun2021 string_sub_test example #12 : FAILED - result does NOT match standard this is a documented fault, worded to correct it at some time in future 12Jun2021 str_removeExtra_whitespace_test - shows possible errors ibase optr? 24*********************24 #**************************** # Table of contents, generated with : # $ grep "^#] " "$d_Qndfs""code development/strings- develop.txt" | sed 's/^#\]/ /' # too large for here! 30Nov2021 will create separate file later.... ********************* 08********08 #] current status of tests +-------------------+ [in, is] tests of strings : 02Dec2021 0/8 ind tstSimple chrL_anyIn_str_test 02Dec2021 11/11 tstSimple chrA_anyIn_strA_test 02Dec2021 7/8 tstSimple str_isOf_chrSet_test 02Dec2021 7/8 tstSimple str_option_isOf_chrSet_test 02Dec2021 4/6 tstSimple subL_anyIn_lst_test +-------------------+ string cleanup : 02Dec2021 2/6 ind tstSimple str_removeExtra_SpcTab_test 02Dec2021 0/6 ind tstSimple strURL_clean_test +-------------------+ convert strings [to, from] other types : 02Dec2021 1/6 quote tstSimple strL_quoteUnifyForHostCmd_test +-------------------+ Unicode tests : 02Dec2021 0/6 tstSimple str_to_unicodeL_test +-------------------+ [self-return, file embedded] executables : 02Dec2021 0/4 tstSimple strL_to_strExecuteMirror_test 02Dec2021 0/6 ind tstSimple strLL_to_strExecuteMirror_test Legend : 0 typically NO [screen output, log file]? ind optr works on examples, but tst_simple doesn't work str_removeExtra_SpcTab_test doesn'loaddefs properly z special table output, need to upgrade as can't see if worked opp ?op_parameter nop ?not an operation quote chr_quote causes problem 02Dec2021 split[Separate,[At[Front,End]], ] - problem was reversed args 08********08 #] 30Nov2021 string tests 'strings/strings- testsGroup.ndf' fix the group testing approach -> fast checking of many optrs!!! +-----+ olde code # 30Nov2021 olde code % if there are NO whitespce left, return a space ; IF (= null result) THEN null ELSE result ENDIF non_whitespace_list := NOT str_isSpcTabs Str ; % find initial whitespace then ending whitespace ; FOR i WITH tell gage shape str DO IF (= l non_whitespace_list@i) THEN first_chr := i ; EXIT null ; ENDIF ; ENDFOR ; FOR i WITH (reverse tell gage shape str) DO IF (= l non_whitespace_list@i) THEN last_chr := i ; EXIT null ; ENDIF ; ENDFOR ; in_white := o ; % retain only the first whitespace in a sequence of whitespaces ; FOR i WITH (first_chr + tell (last_chr - first_chr)) DO IF non_whitespace_list@i THEN IF in_white THEN in_white := o ; ENDIF ; ELSE IF NOT in_white THEN non_whitespace_list@i := l ; in_white := l ; ENDIF ; ENDIF ; ENDFOR ; IF (= null result) THEN null ELSE result := first (non_whitespace_list sublist Str) ENDIF IF (= null result) THEN null ELSE first (non_whitespace_list sublist Str) ENDIF # t04 is a problem qnial> a := str_removeExtra_SpcTab '' >> apparently null? qnial> diagram a +-+ | | +-+ qnial> a := str_removeExtra_SpcTab '' (apparently a space) qnial> diagram a +-+ | | +-+ qnial> a = [' '] o qnial> a = solitary ' ' o qnial> a = single a o qnial> single a o-+ | | +-+ qnial> frst a ?undefined identifier: FRST <***> A qnial> first a (space?) qnial> b := first a (space?) qnial> type b (space?) qnial> diagram b (space?) qnial> solitary a +-+ | | +-+ qnial> a = solitary b l QNial dictionary character qnial> a = ' ' l >> NUTSS!!! why a space??? qnial> c := non_whitespace_list := NOT str_eachChrIs_white '' >> apparently null qnial> c = null l >> confirmed qnial> d := non_whitespace_list sublist null >> apparently null qnial> d = null l >> confirmed >> save to '0_QNial special problems.txt' 08*******08 23Nov2021 post-massive symbol name changes : upgrade to new format & run 'strings- tests.ndf' 08*******08 01Oct2021 strL_cutBy_chrL[,_test] - messed up shapes >> WOW! that was tough, but it finally worked. QNial manual : The transformer TWIG transforms an operation f into an operation that applies f to every simple array in the nested structure of A. The resulting operation is called the TWIG transform of f. The result of applying the TWIG transform of f to A has the same shape as A. If f maps simple arrays to simple arrays of the same shape, the result has the same structure as A. Definition TWIG IS TRANSFORMER f OPERATION A {­ IF simple A THEN f A ELSE EACH (TWIG f) A ENDIF } >> this should work with strL_cutBy_chrL, but doesn't seem so >> what is "simple"? The operation simple tests whether or not an array has all atomic items or is an empty array. The result is true if the array is simple and false if it is not. Definition simple IS OPERATION A (EACH single A = A) >> again, this is what I want, so why is my code wrong? Maybe cart? seems unlikely...try it > what is "single" The operation single returns an array with no axes holding A as its only item. The result is an array with shape Null and is said to be a single . Definition single IS OPERATION A {­ Null reshape solitary A } >> ??? I don't understand what this does in detail - do only atoms pass A = single A? NYET!!! -->[nextv] 5 = single 5 l -->[nextv] 'hello' = single 'hello' o >> this is the problem!!?? I need to dene transformer TWAG that does what I want +-----+ Working notes and attempts # loaddefs link d_Qndfs 'Qndfs optr symbol changes.ndf' # examples c05 := 3 2 reshape 'My friend, what $ do you have for 5 hombres?' 'Los amigos, who has $, and who has amigos?' 'How much $ did you make?' 'If you can sing 5 songs, start a fan club.' 'No matter, 555 is a big number.' 'Nobody knows anything, right?' ; d05 := ' ,$5' ; # strL_cutBy_chrL c05 d05 # works but str to chr results in simple arrays: boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL))) ; # ?conform : boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT TWIG = strL))) ; boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT TWIG EACHRIGHT = strL))) ; boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT EACHRIGHT TWIG = strL))) ; # olde code strL_cutBy_chrL IS OP strL chrL { LOCAL boolLL ; boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL))) ; boolLL EACHBOTH booleanL_cut_list strL } boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL))) ; boolLL := OR cols boolLL ; boolLL := NOT (cols mix (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL)) ; % argAry := (boolAry EACHBOTH cart (cols strL)) ; argAry := (shape strL) split (boolAry EACHBOTH cart (cols strL)) ; % argAry := (boolAry EACHBOTH cart (cols strL)) ; argAry := 1 0 split (boolAry EACHBOTH cart (cols strL)) ; argAry := 0 split (boolAry EACHBOTH cart (cols strL)) ; boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL))) ; boolLL := NOT (EACH EACHALL OR (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL)) ; argAry := (shape strL) reshape link (boolAry EACHBOTH cart (cols strL)) ; argAry := 1 split (boolAry EACHBOTH cart (cols strL)) ; argAry := 1 0 blend (boolAry EACHBOTH cart (cols strL)) ; argAry := 2 0 blend (boolAry EACHBOTH cart (cols strL)) ; argAry := 2 1 blend (boolAry EACHBOTH cart (cols strL)) ; argAry := 2 1 0 blend (boolAry EACHBOTH cart (cols strL)) ; argAry := (reverse shape strL) reshape link (boolAry EACHBOTH cart (cols strL)) ; boolLL := NOT (EACH EACHALL OR (cols mix (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL))) ; boolLL := NOT EACH EACHALL OR boolTab ; boolLL := NOT EACH EACHALL OR (EACH cols boolTab) ; boolLL := NOT OR EACH EACH cols boolTab ; boolAry := boolLL ; argAry := transpose ((reverse shape strL) reshape link (boolAry EACHBOTH cart (cols strL))) ; EACH booleanL_cut_list argAry argAry := boolAry EACHBOTH cart (EACH solitary strL) ; >> WOW! that was tough, but it finally worked. +-----+ ?.. Boolll := not ( EACH EACHALL or ( cols mix ( Chrl EACHLEFT EACHRIGHT EACHRIGHT = Strl ) ) ) -->[nextv] +--------------------------------------------+------------------------------------------+ |llolllllllollllolollolllollllolllolollllllll|lllolllllllolllolllollolllolllolllolllllll| +--------------------------------------------+------------------------------------------+ |lllllllllollllllllllllllllllllllllllllllllll|llllllllllollllllllllollllllllllllllllllll| +--------------------------------------------+------------------------------------------+ |llllllllllllllllolllllllllllllllllllllllllll|llllllllllllllllllllolllllllllllllllllllll| +--------------------------------------------+------------------------------------------+ |llllllllllllllllllllllllllllllllllolllllllll|llllllllllllllllllllllllllllllllllllllllll| +--------------------------------------------+------------------------------------------+ ?.. Argary := Boolll EACHBOTH pair Strl -->[nextv] ?conform ?.. Boolll EACHBOTH booleanl_cut_list Strl -->[nextv] strL +--------------------------------------------+------------------------------------------+ |My friend, what $ do you have for 5 hombres?|Los amigos, who has $, and who has amigos?| +--------------------------------------------+------------------------------------------+ |My friend, what $ do you have for 5 hombres?|Los amigos, who has $, and who has amigos?| +--------------------------------------------+------------------------------------------+ |My friend, what $ do you have for 5 hombres?|Los amigos, who has $, and who has amigos?| +--------------------------------------------+------------------------------------------+ -->[nextv] TWIG (gage shape) boolLL strl +-----+-----+ |44 42|44 42| |44 42|44 42| |44 42|44 42| |44 42| | +-----+-----+ >> why does boolLL have 4 rows? OK as it's supposed to OK : There are 4 chrs in d05 := ' ,$5' ; I didn't "combine" boolLL!! 08*******08 10Sep2021 string operators [rename, re-order args] part of overall "clean-up" of nomenclature of my QNial operators etc see "$d_Qroot""code develop_test/strings/210910 string operators [rename, re-order args].txt" 03Sep2021 [strings, fileops, QNial setup].ndf - comment for commenting out old optr names : >> special "mini-Project" 08********08 29Sep2021 fix string [symbol, arg]s - whole-word search-replace in d_Qndfs see unsorted list of substitutions : "$d_Qroot""210929 QNial operator names - substitutions.txt" $ cat "$d_Qroot""210929 QNial operator names - substitutions.txt" | sort 30Sep2021 string optrs - consolidation by generalisation #] strL_cutBy_chrL IS OP strL chrL - cut each strL by all chrL, omitting chrL # formal tests - see link d_Qtest 'strings/strings- tests.ndf' # 11Jun2021 inital strL_cutBy_chrL IS OP strL chrL { booleanL_cut_list (NOT (chrL EACHLEFT EACHRIGHT EACHRIGHT = strL)) str } >> this is really cool, I'm learning something! # qnial> strL +-----------------------------------------------+------------------------------------------+ |My friend, what $ do you have for four hombres?|Los amigos, who has $, and who has amigos?| +-----------------------------------------------+------------------------------------------+ qnial> a := ' ,$5' ,$5 qnial> `5 EACHRIGHT = strL oo qnial> `5 TWIG = strL +-+--+ |l|oo| +-+--+ qnial> `5 TWIG = first strL lo qnial> `5 TWIG = (first strL) lo qnial> first strL My friend, what $ do you have for four hombres? qnial> `5 = (first strL) o qnial> z1 := a EACHLEFT EACHRIGHT = (first strL) +-----------------------------------------------+-----------------------------------------------+------------- |ooloooooooloooololooloooloooolooolooooloooooooo|ooooooooolooooooooooooooooooooooooooooooooooooo|ooooooooooooo +-----------------------------------------------+-----------------------------------------------+------------- ----------------------------------+-----------------------------------------------+ oooloooooooooooooooooooooooooooooo|ooooooooooooooooooooooooooooooooooooooooooooooo| ----------------------------------+-----------------------------------------------+ qnial> EACHALL = z1 llolllllloollllooollolllollllolllollllollllllll qnial> EACHALL or z1 ooloooooolloooolllooloooloooolooolooooloooooooo qnial> booleanL_cut_list (NOT z2) (first strL) +--+------+----+--+---+----+---+----+--------+ |My|friend|what|do|you|have|for|four|hombres?| +--+------+----+--+---+----+---+----+--------+ qnial> z3 := a EACHLEFT EACHRIGHT = (second strL) +------------------------------------------+------------------------------------------+----------------------- |oooloooooooloooloooloolooolooolooolooooooo|oooooooooolooooooooooloooooooooooooooooooo|ooooooooooooooooooooloo +------------------------------------------+------------------------------------------+----------------------- -------------------+------------------------------------------+ ooooooooooooooooooo|oooooooooooooooooooooooooooooooooooooooooo| -------------------+------------------------------------------+ qnial> z4 := EACHALL or z3 oooloooooollooolooollllooolooolooolooooooo qnial> booleanL_cut_list (NOT z4) (second strL) +---+------+---+---+---+---+---+-------+ |Los|amigos|who|has|and|who|has|amigos?| +---+------+---+---+---+---+---+-------+ qnial> z10 := chrL EACHLEFT EACHRIGHT EACHRIGHT = strL +--------------------------------------------------------------------------------------------+---------------- |+-----------------------------------------------+------------------------------------------+|+--------------- ||ooloooooooloooololooloooloooolooolooooloooooooo|oooloooooooloooloooloolooolooolooolooooooo|||ooooooooolooooo |+-----------------------------------------------+------------------------------------------+|+--------------- +--------------------------------------------------------------------------------------------+---------------- ----------------------------------------------------------------------------+--------------------------------- --------------------------------+------------------------------------------+|+-------------------------------- oooooooooooooooooooooooooooooooo|oooooooooolooooooooooloooooooooooooooooooo|||oooooooooooooooolooooooooooooooo --------------------------------+------------------------------------------+|+-------------------------------- ----------------------------------------------------------------------------+--------------------------------- -----------------------------------------------------------+-------------------------------------------------- ---------------+------------------------------------------+|+-----------------------------------------------+- ooooooooooooooo|oooooooooooooooooooolooooooooooooooooooooo|||ooooooooooooooooooooooooooooooooooooooooooooooo|o ---------------+------------------------------------------+|+-----------------------------------------------+- -----------------------------------------------------------+-------------------------------------------------- ------------------------------------------+ -----------------------------------------+| ooooooooooooooooooooooooooooooooooooooooo|| -----------------------------------------+| ------------------------------------------+ qnial> z11 := EACHALL or z10 +-----------------------------------------------+------------------------------------------+ |ooloooooolloooolllooloooloooolooolooooloooooooo|oooloooooollooolooollllooolooolooolooooooo| +-----------------------------------------------+------------------------------------------+ qnial> (NOT z11) EACHBOTH booleanL_cut_list strL +----------------------------------------------+----------------------------------------+ |+--+------+----+--+---+----+---+----+--------+|+---+------+---+---+---+---+---+-------+| ||My|friend|what|do|you|have|for|four|hombres?|||Los|amigos|who|has|and|who|has|amigos?|| |+--+------+----+--+---+----+---+----+--------+|+---+------+---+---+---+---+---+-------+| +----------------------------------------------+----------------------------------------+ 08*******08 11Jun2021 rewrote some optrs using new optr booleanL_cut_list str_cutBy_chr IS OP chr str { booleanL_cut_list (chr EACHRIGHT in str) str } Run d_Qtest 'strings/strings- tests.ndf' - which has modified str_splitAtFront_subStr +-----+ ************************************* Summary of test results : /media/bill/Dell2/Website - raw/Qnial/code develop_test/strings/210612 13h34m15s alltest strings.txt, date= 210612 13h34m # str_to_unicodeList_test example #1 : OK - result matches standard # str_to_unicodeList_test example #2 : OK - result matches standard # str_to_unicodeList_test example #3 : OK - result matches standard # str_to_unicodeList_test example #4 : OK - result matches standard # str_to_unicodeList_test example #5 : OK - result matches standard # str_to_unicodeList_test example #6 : OK - result matches standard # str_replace_subStr_test example #1 : OK - result matches standard # str_replace_subStr_test example #2 : OK - result matches standard # str_replace_subStr_test example #3 : OK - result matches standard # str_replace_subStr_test example #4 : OK - result matches standard # str_replace_subStr_test example #5 : OK - result matches standard # str_replace_subStr_test example #6 : OK - result matches standard # str_replace_subStr_test example #7 : OK - result matches standard # str_replace_subStr_test example #8 : OK - result matches standard # str_replace_subStr_test example #9 : OK - result matches standard # str_replace_subStr_test example #10 : OK - result matches standard # str_replace_subStr_test example #11 : OK - result matches standard # str_replace_subStr_test example #13 : OK - result matches standard # str_splitAtFront_subStr_test example #1 : OK - result matches standard # str_splitAtFront_subStr_test example #2 : OK - result matches standard # str_splitAtFront_subStr_test example #3 : OK - result matches standard # str_splitAtFront_subStr_test example #4 : OK - result matches standard # str_splitAtFront_subStr_test example #5 : OK - result matches standard # str_splitAtFront_subStr_test example #6 : OK - result matches standard # str_splitAtFront_subStr_test example #7 : OK - result matches standard # str_splitAtFront_subStr_test example #8 : OK - result matches standard # str_splitLftRgtTo_midIndxs_StrList_test example #1 : OK - result matches standard # str_splitLftRgtTo_midIndxs_StrList_test example #2 : OK - result matches standard # str_splitLftRgtTo_midIndxs_StrList_test example #3 : OK - result matches standard # str_splitLftRgtTo_midIndxs_StrList_test example #4 : OK - result matches standard # str_splitLftRgtTo_midIndxs_StrList_test example #5 : OK - result matches standard # str_replace_subStr_test example #12 : FAILED - result does NOT match standard +-----+ >> 12Jun2021 OK except string_sub_test - which now is str_replace_subStr (renamed in tests) This has to be fixed NOW! Many optrs may depend on it Add it to pre-declared optrs in strings.ndf >> NYET! - this is a documented fault, worded to correct it at some time +-----+ # olde code str_cutBy_chr IS OP chr Str { LOCAL Str_list ; Str_list := link [solitary first, EACH rest rest] (str_splitBy_str (string chr) Str) ; % remove nulls from repeated occurences of chr ; (NOT (null EACHRIGHT = Str_list)) sublist Str_list } str_cutBy_subStr IS OP str subStr { LOCAL in_subStr i j subStr_length str_length subStr_tell head tail result_back string_block ; subStr_length str_length := EACH tally subStr str ; string_block := solitary str ; IF subStr_length < str_length THEN subStr_tell := tell subStr_length ; FOR j WITH tell (- str_length subStr_length + 1) DO in_subStr := l ; FOR i WITH subStr_tell DO IF subStr|[i] ~= str|[j+i] THEN in_subStr := o ; EXIT null; ENDIF ; ENDFOR ; IF in_subStr = l THEN head tail := [tell j, j + subStr_length + (tell (str_length - j - subStr_length))] EACHLEFT choose str ; result_back := subStr_splitWith_subStr subStr tail ; string_block := (head subStr (first result_back)) link (rest result_back) ; EXIT null; ENDIF ; ENDFOR ; ENDIF ; string_block } # str_replace_subStr 'and' 'jump' 'This and that and the other thing' # 25May2021 olde versione str_replace_subStr IS OP sub_old sub_new str { LOCAL i_str i_sub_old len_str len_sub_old textnew ; % ; IF (NOT OR (sub_old sub_new EACHLEFT = null)) THEN len_sub_old := gage shape sub_old ; len_str := gage shape str ; i_str i_sub_old := 0 0 ; textnew := null ; WHILE (i_str < len_str) DO WHILE (AND ( (i_sub_old + i_str) < len_str) (i_sub_old < len_sub_old) (= sub_old@i_sub_old str@(i_str + i_sub_old) ) ) DO i_sub_old := i_sub_old + 1 ; ENDWHILE ; %write link '[i_str,i_sub_old]= ' (string i_str) ', ' (string i_sub_old) ; IF (len_sub_old = i_sub_old) THEN textnew := link textnew sub_new ; i_str := i_str + len_sub_old - 1 ; ELSEIF (0 <= i_sub_old) THEN textnew := link textnew str@i_str ; ELSE write fault '?str_replace_subStr - i_sub_old is anomalous' ; ENDIF ; %write textnew ; i_str := i_str + 1 ; i_sub_old := 0 ; ENDWHILE ; % ; ELSE textnew := null ; ENDIF ; % ; IF (= null textnew) THEN str ELSE textnew ENDIF } 08********08 29Sep2021 fix string [symbol, arg]s +-----+ $ find "$d_Qndfs" -type f -name "*.ndf" | grep --invert-match 'z_Archive' | grep --invert-match 'z_Old' | tr \\n \\0 | xargs -0 -ILINE grep -w -i --with-filename --line-number "str_to_phrases1" LINE /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/QNial setup.ndf: str_to_phrases1 IS OP str { null } /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/strings.ndf: Day Month Year := 2 3 4 EACHLEFT pick (str_to_phrases1 emlDate) ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/strings.ndf: Day Month Year HourMinuteSecond := 2 3 4 5 EACHLEFT pick (str_to_phrases1 emlDate) ; >> changed 'str_to_phrases1' to 'str_to_phrases' +-----+ str_findPositionsOf_subStr - definition is missing? NYET - more precisely defined by : str_find1stChrOf_subStr str_findAllChrsOf_subStr $ find "$d_Qndfs" -type f -name "*.ndf" | grep --invert-match 'z_Archive' | grep --invert-match 'z_Old' | tr \\n \\0 | xargs -0 -ILINE grep -w -i --with-filename --line-number "str_findPositionsOf_subStr" LINE /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/strings.ndf:1299:# 06Sep2021 for deletion : find_string IS str_findPositionsOf_subStr /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/strings.ndf:1819: subPosns := str_findPositionsOf_subStr str subStr ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/strings.ndf:2552: flag_pairs := (= 0 (mod (gage shape str_findPositionsOf_subStr str strStrt) 2)) ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/strings.ndf:2558: fronts rears := strStrt strEndr EACHLEFT str_findPositionsOf_subStr str ; >> all changed in strings.ndf? +-----+ $ find "$d_Qndfs" -type f -name "*.ndf" | grep --invert-match 'z_Archive' | grep --invert-match 'z_Old' | tr \\n \\0 | xargs -0 -ILINE grep -w -i --with-filename --line-number "find_string" LINE /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video production/strings.ndf:74:IF not in "FIND_STRING (EACH first symbols 0) THEN /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video production/strings.ndf:143:# find_string Substr Str /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video production/strings.ndf:153:find_string IS OPERATION Substr Str { /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video production/strings.ndf:160:find_string IS OPERATION Substr Str /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video production/strings.ndf:183:'the' EACHRIGHT find_string ['There is a way','and you have','it'] /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video production/strings.ndf:185:# 'the' find_string 'Is that their cat?' /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video capture/strings.ndf:74:IF not in "FIND_STRING (EACH first symbols 0) THEN /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video capture/strings.ndf:143:# find_string Substr Str /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video capture/strings.ndf:153:find_string IS OPERATION Substr Str { /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video capture/strings.ndf:160:find_string IS OPERATION Substr Str /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video capture/strings.ndf:183:'the' EACHRIGHT find_string ['There is a way','and you have','it'] /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/video production/video capture/strings.ndf:185:# 'the' find_string 'Is that their cat?' /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:19:IF not in "FIND_STRING (EACH first symbols 0) THEN /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:165: ELSEIF find_string 'mailer-daemon' emails@i THEN dups@i := o ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:166: ELSEIF find_string 'postmaster' emails@i THEN dups@i := o ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:167: ELSEIF find_string 'ijcnn' emails@i THEN dups@i := o ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:168: ELSEIF find_string '...' emails@i THEN dups@i := o ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:169: ELSEIF find_string '>' emails@i THEN dups@i := o ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:170: ELSEIF find_string 'howell' emails@i THEN dups@i := o ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf:171: ELSEIF find_string 'bill@billh' emails@i THEN dups@i := o ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf:124: (find_string 'dell64' winStr) + 7 drop winStr /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf:129:# qnial> find_string 'dell64' (first host_result link 'wmctrl -l | grep "windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf" ') /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/strings.ndf:1299:# 06Sep2021 for deletion : find_string IS str_findPositionsOf_subStr /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/Thunderbird - delete binary attachments.ndf:7: The operation uses the library operations "stripblanks" and "find_string". /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/conference guides - update inserts.ndf:106:# 24Sep2018 find_string returns a LIST! /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/conference guides - update inserts.ndf:113: i_str := first find_string ':@file-insert@:' str + (gage shape ':@file-insert@:') ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/port_translate_Matlab_to_QNial.ndf:249: res_start res_end := gage (('function' '=') EACHLEFT find_string func_declare) ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/port_translate_Matlab_to_QNial.ndf:293: woird5_strt := find_string woird5 line_matlab ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/port Matlab to Nial.ndf:5: The operation uses the library operations "stripblanks" and "find_string". /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email Thunderbird - delete binary attachments.ndf:20: The operation uses the library operations "stripblanks" and "find_string". /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email - extract, sort, cull addresses from text.ndf:155:IF not in "FIND_STRING (EACH first symbols 0) THEN /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email - extract, sort, cull addresses from text.ndf:220: at_addresses := find_string '@' line_in ; % 12Sep2016 was find ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email - extract, sort, cull addresses from text.ndf:445: IF (~= null (hyph_count := (gage shape (find_string '-' line)))) /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email - extract, sort, cull addresses from text.ndf:449: IF (~= null (amp_count := (gage shape (find_string '@' line)))) /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/emails from conf papers.ndf:52:IF not in "FIND_STRING (EACH first symbols 0) THEN /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email - ctry&address from list.ndf:39:IF not in "FIND_STRING (EACH first symbols 0) THEN /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email - ctry&address from list.ndf:67: posn_period := find_string '.' line_in ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/email/email - ctry&address from list.ndf:93: posn_period := find_string '.' em_chrs ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/symbols/symbols fix.ndf:66: POI_posn := find_string (each string '(POI') symChrList ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/symbols/symbols fix.ndf:168: POI_posns := find_string (each string '(POI') chrList ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/matrix operations - symbolic & real-valued.ndf:1351: sliceNum := gage shape find_string '|[' RHS ; /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/matrix operations - symbolic & real-valued.ndf:1356: slice_addr := 0 choose find_string '|[' RHS ; >> ouch!! that's a lot to fix! >> most likely need : str_find1stChrOf_subStr >> careful - arguments are probably re-ordered "$d_Qndfs""video production/video production/strings.ndf" >> PROBLEM, uses its own strings.ndf : worry about this later "$d_Qndfs""IJCNN 2013 Dallas TX/IJCNN2013 Dallas - Get email addresses in dumped undeliverables.ndf" IF not in "FIND_STRING (EACH first symbols 0) THEN set "sketch; loaddefs (link QNial_root 'niallib/FINDSTRI.NDF') 0; ENDIF; >> changed to 'Qnial_bag/niallib/' find_string IS OPERATION Substr Str { Position := first Substr findall Str ; Substr in ( tally Substr EACHRIGHT take ( Position EACHLEFT drop Str ) ) } >> this is a test returning boolean # 29Sep2021 see find_strings in "$d_ndfs""Qnial_bag/niallib/FINDSTRI.NDF" - simpler approach # it would be interesting to compare the performance of the two approaches >> just leave it as is!! "$d_Qndfs""windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf" winIDorTitle_get_winTitle IS OP winIDorTitle { LOCAL winSpecsOne winStr ; winSpecsOne := host_result link 'wmctrl -l | grep "' winIDorTitle '" ' ; IF (= null winSpecsOne) THEN fault '?winIDorTitle_get_winTitle no windows' ELSEIF (= 1 (gage shape winSpecsOne)) THEN winStr := first winSpecsOne ; (find_string 'dell64' winStr) + 7 drop winStr ELSE fault '?winIDorTitle_get_winTitle multiple winIDs or fault' ENDIF } >> This is important and has to work. It seems to requ a position, not a boolean? qnial> find_string 'dell64' (first host_result link 'wmctrl -l | grep "windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf" ') ?undefined identifier: FIND_STRING <***> 'dell64' ( FIRST Try : qnial> str_find1stChrOf_subStr 'dell64' (first host_result link 'wmctrl -l | grep "windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf" ') $ wmctrl -l 0x0ac00003 3 dell64 windows [open,…[posn,size].ndf - /media/bill/Dell2/Website - raw/Qnial/MY_NDFS - Geany (new instance) $ wmctrl -l | grep "windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf" ~ >>can't work as geany replaces middle of fname with '...', and puts the directory AFTER the fname! >> for now, just change 'find_string' to 'str_find1stChrOf_subStr', reverse args, and fix later... "$d_Qndfs""Thunderbird - delete binary attachments.ndf" >> ignore, as is just a comment and I don't use this file anymore "$d_Qndfs""conference guides - update inserts.ndf" >> changed 'find_string' to 'str_find1stChrOf_subStr', reversed args >> worry about effects when I next use this "$d_Qndfs""port_translate_Matlab_to_QNial.ndf" >> changed 'find_string' to 'str_find1stChrOf_subStr', reversed args >> worry about effects when I next use this "$d_Qndfs""port Matlab to Nial.ndf" >> removed comment 'The operation uses the library operations "stripblanks" and "find_string".' "$d_Qndfs""email/email Thunderbird - delete binary attachments.ndf" >> removed comment 'The operation uses the library operations "stripblanks" and "find_string".' "$d_Qndfs""email/email - extract, sort, cull addresses from text.ndf" >> changed 'find_string' to 'str_find1stChrOf_subStr' , reversed args >> changed 'find_strings' to 'subStrL_anyIn_str' >> worry about effects when I next use this "$d_Qndfs""email/emails from conf papers.ndf" >> removed (not used) : # This is loaded with every QNial setup : IF not in "FIND_STRING (EACH first symbols 0) THEN set "sketch; loaddefs "/media/bill/SWAPPER/Qnial/niallib/FINDSTRI.NDF 0; ENDIF; "$d_Qndfs""email/email - ctry&address from list.ndf" >> removed (was used) : # This is loaded with every QNial setup : IF not in "FIND_STRING (EACH first symbols 0) THEN set "sketch; loaddefs "/media/bill/SWAPPER/Qnial/niallib/FINDSTRI.NDF 0; ENDIF; >> changed 'find_string' to 'str_find1stChrOf_subStr' , reversed args "$d_Qndfs""symbols/symbols fix.ndf" >> changed 'find_string' to 'str_find1stChrOf_subStr' , reversed args >> but doesn't look right!?!?!? POI_posn := str_find1stChrOf_subStr symChrList (each string '(POI') ; POI_posns := str_find1stChrOf_subStr chrList (each string '(POI') ; >> worry about effects when I next use this "$d_Qndfs""matrix operations - symbolic & real-valued.ndf" >> changed 'find_string' to 'subStr_in_str', NOT reversed args >> worry about effects when I next use this +--+ old pre-declarations in 'QNial setup.ndf' : # 29Sep2021 - unnecessary after fixing d_Qndfs : # should be OK? str_splitByChrsNotAllowedTo_words IS OP str chrSet { null } # discontined? hhmmss_elapsed IS OP t_start_sec { null } # changed to str_prepadWith_chr lead_chrs IS OP len chr str { null } # find_Howell doesn't work? - had been commented out (== ary_find1st_subAry) month_number IS OP month_name { null } # should be OK? sec_from_timestamp IS null # should be OK? str_cutBy_subStr IS OP str subStr { null } # should be OK? str_find1stChrOf_subStrL IS OP str subStrL { null } # should be OK? str_splitAtFront_subStr IS OP str subStr { null } # should be OK? timestamp_YYMMDD_HMS IS null # should be OK? write_debug IS OP AAA { null } +--+ template : >> removed (was used) : # This is loaded with every QNial setup : IF not in "FIND_STRING (EACH first symbols 0) THEN set "sketch; loaddefs "/media/bill/SWAPPER/Qnial/niallib/FINDSTRI.NDF 0; ENDIF; >> changed 'find_string' to 'str_find1stChrOf_subStr' , reversed args >> changed 'find_strings' to 'subStrL_anyIn_str' , NOT reversed args >> worry about effects when I next use this +-----+ for deletion (replace optr_name) : # 06Sep2021 for deletion : str_splitBy_char IS str_cutBy_chr $ find "$d_Qndfs" -type f -name "*.ndf" | grep --invert-match 'z_Archive' | grep --invert-match 'z_Old' | tr \\n \\0 | xargs -0 -ILINE grep -w -i --with-filename --line-number "str_splitBy_char" LINE economics, markets/winURL yahoo finance news download.ndf:96: minLess := tonumber (first str_splitBy_char ` newsTime) ; economics, markets/winURL yahoo finance news download.ndf:98: hurLess := tonumber (first str_splitBy_char ` newsTime) ; economics, markets/winURL yahoo finance news download.ndf:228:# str_splitBy_char `, 'Treasuries Reprieve Buoys Emerging Markets, but Buyers Are Picky' economics, markets/winURL yahoo finance news download.ndf:334: minLess := tonumber (first str_splitBy_char ` line) ; economics, markets/winURL yahoo finance news download.ndf:336: hurLess := tonumber (first str_splitBy_char ` line) ; webSite/webSite maintain [menu, header, footer, body] links, TableOfContents.ndf:556:# str_splitBy_char 'boolean list cut#into alternate [o, l] sublists' `# webSite/webSite maintain [menu, header, footer, body] links, TableOfContents.ndf:557:# str_splitBy_char 'boolean list cut into alternate [o, l] sublists' `# bank statements reformat.ndf:181:# dater tranType := str_splitBy_char quack chr_tab bank statements reformat.ndf:182:# day month year := str_splitBy_char dater ` >> changed 'str_splitBy_char' to 'str_cutBy_chr' , reversed args (they were already like that) +-----+ NUTS! This is takes WAAAYYY too long!!! just do a proper word-style replacement for every optr_name then fix the argument [number, order] +-----+ Create a list of optr_name substitutions # 06Sep2021 for deletion : string_cut_by_char str_cutBy_chr str_splitBy_subStr str_cutBy_chr str_splitBy_subStr str_cutBy_subStr str_splitBy_str str_cutBy_subStr str_splitWith_subStr str_splitAtFront_subStr string_splitWith_string str_splitAtFront_subStr string_split_by_string str_splitAtFront_subStr $ find "$d_Qndfs" -type f -name "*.ndf" | grep --invert-match 'z_Archive' | grep --invert-match 'z_Old' | tr \\n \\0 | xargs -0 -ILINE grep -w -i --with-filename --line-number "# 06Sep2021 for deletion :" LINE /media/bill/Dell2/Website - raw/Qnial/MY_NDFS/ QNial setup.ndf:1141:# 06Sep2021 for deletion : array_findAll_subArray ary_findAll_subAry findAll_Howell ary_findAll_subAry array_replace_subArray ary_replace_subAry sortup_cull_listCollection lists_sortupCullOn1st strings.ndf:819:# 06Sep2021 for deletion : string_apos str_surroundWithApos string_apos_tabs str_surroundWithAposAddTab char_nonblank_before str_lastNonWhiteChr_position find_chrs_special str_findChrsInSet in_string subStr_in_str string_in subStr_in_str find_string str_find1stChrOf_subStr string_in_file str_in_path is_blankstring str_isBlank is_whiteLine str_isWhite whitespace_flags str_eachChrIs_white str_prepadWith_chr str_prepadWith_chr string_postpad str_postpadWith_chr_toLength strip_chrs str_strip_chr stripblanks str_stripAllSpacesQ strip_allBlanks str_stripAllSpaces strip_extra_whitespace str_removeExtra_whitespace string_sub str_replace_subStr strOld_sub_strNew_in_strFull str_replaceAll_strOld strFull_strFront_extract_strEnd str_extractPast_strFront str_extractFrom_strFront str_extractPast_strFront string_wide str_reduceTo_width str_splitLftRgtTo_Indxs_StrList str_splitLftRgtTo_midIndxs_StrList string_split_by_string_with_sub str_replaceSubTo_strList cuttext str_splitByWidthTo_strList str_replaceIn_strList strL_replace_str splitStringTo_words str_splitWithChrsAllowedTo_words str_splitTo_words str_splitWithChrsAllowedTo_words replaceWordsubStr_in_str word_replaceIn_str array_to_str_sep atomList_to_str string_base array_atomsTo_strs array_of_strings array_leafsTo_strs tostring array_to_str towords str_to_phrases string_the_numList numList_to_str strings_between str_extractBetween_StrtEnd +-----+ loading str_splitSeparate_subStr ?undefined identifier: RESULT_BACK := STR_SPLITWITH_SUBSTR <***> TAIL SUBSTR ; loading str_splitAtEnd_subStr loading str_splitAtFront_subStr ?undefined identifier: STRL := STR_SPLITWITH_SUBSTR <***> STR SUBSTR ; loading str_splitSeparate_subStrL ?undefined identifier: STRL EACHLEFT STR_SPLITSEPARATE_SUBSTR <***> SUBSTR ; ENDFOR loading str_splitAtFront_subStrL loading str_splitLftRgtTo_midIndxs_StrList ?undefined identifier: STRRGT EACHRIGHT STR_SPLITWITH_SUBSTR <***> ( STR_SPLITWITH_SUBSTR +-----+ # olde code str_cutBy_subStr IS OP str subStr { LOCAL boolL chrPosns str_length subPosns subStr_length ; subStr_length str_length := EACH tally subStr str ; subPosns := str_findPositionsOf_subStr str subStr ; chrPosns := link (subPosns EACHLEFT + (tell subStr_length)) ; boolL := str_length reshape l ; boolL#(chrPosns) := o ; booleanL_cut_list boolL str } 08********08 10Sep2021 list of checks for each file >> This just complicates everything too much! just do one correction per file, iterate over corrections bank statements reformat.ndf: str_cutBy_chr versus str_splitBy_char, argOrder economics, markets/winURL yahoo finance news download.ndf: str_cutBy_chr versus str_splitBy_char, argOrder email/email - ThunderBlog.ndf: string_sub IS str_replace_subStr, argOrder, delete old names email - mass response processing/210506 15h00m34s email - split Thunderbird email folder.ndf: string_sub IS str_replace_subStr, argOrder, delete old names email - mass response processing/email - split Thunderbird email folder.ndf: string_sub IS str_replace_subStr, argOrder, delete old names gnuplot.ndf: string_sub IS str_replace_subStr, argOrder, delete old names QNial setup.ndf: string_sub IS str_replace_subStr, argOrder, delete old names MY_NDFS/QNial setup.ndf: str_cutBy_chr versus str_splitBy_char, argOrder webSite/webSite maintain [menu, header, footer, body] links, TableOfContents.ndf: str_cutBy_chr versus str_splitBy_char, argOrder 08********08 03Sep2021 loaddef errors [QNial setup, strings, fileops, windows [open, close, ID, title]].ndf : phrase to insert intdf files to denote where changes must be made # 06Sep2021 for deletion : I might as well major cleanup while fixing re-ordering of args for strings.ndf optrs (started 03Sep2021 below) +-----+ 'QNial setup.ndf' : partial list of re-ordered optrs? : +-----+ 'QNial setup.ndf' loaddefs errors : ?undefined identifier: NOT ( IN_STRING <***> DEBUG_STRT LINE2 ) ?undefined identifier: NOT ( IN_STRING <***> DEBUG_STRT LINE2 ) ?undefined identifier: EACHRIGHT = ARY_TO_SEARCH <***> ; I_ADDS := ?undefined identifier: ) EACHLEFT FINDALL_HOWELL <***> ARY ; ( ary_findIndxs_commonItems IS OP ary { LOCAL ary ary_unique indxsCommonL keysAll_indxsIn_aryOfLists ; indxsCommonL := ary EACHRIGHT ary_findAll_subAry (cull sortup ary) ; (1 < (EACH (gage shape) indxsCommonL)) sublist indxsCommonL } ?undefined identifier: ; WRITE ARY_FINDINDXS_COMMONITEMS <***> 'my' 'bad' 'ideas' str_cutBy_chr IS OP chr Str { null } # 06Sep2021 for deletion : str_splitBy_char IS str_cutBy_chr # 06Sep2021 for deletion : string_cut_by_char IS str_cutBy_chr # 06Sep2021 for deletion : str_splitBy_subStr IS str_cutBy_chr str_cutBy_subStr IS OP str subStr { null } # 06Sep2021 for deletion : str_splitBy_subStr IS str_cutBy_subStr # 06Sep2021 for deletion : str_splitBy_str IS str_cutBy_subStr str_splitAtFront_subStr IS OP subStr str { null } # 06Sep2021 for deletion : str_splitWith_subStr IS str_splitAtFront_subStr # 06Sep2021 for deletion : string_splitWith_string IS str_splitAtFront_subStr # 06Sep2021 for deletion : string_split_by_string IS str_splitAtFront_subStr str_replace_subStr IS OP sub_old sub_new str { null } # 06Sep2021 for deletion : string_sub IS str_replace_subStr I [renamed, switched argument ordering] : ary_findAll_subAry IS OP ary subAry ary_find1st_subAry IS OP ary subAry +-----+ strings.ndf >>> loading start : strings.ndf ?undefined identifier: TONUMBER ( STR_SPLITBY_CHAR <***> `: HHMMSS ) ?undefined identifier: TONUMBER ( STR_SPLITBY_CHAR <***> `: T_VOICE ) ?undefined identifier: SECOND := STR_SPLITBY_CHAR <***> `: HOURMINUTESECOND ; <<< loading ended : strings.ndf I had already switched ordering : str_splitLftRgtTo_midIndxs_StrList IS OP str strLft strRgt - split str, return [indxs, strLst] !!!***************** >> this may be one cause of webSite link problems? !!!***************** arg positions : Change : str_findChrsInSet IS OP flag_chrs_include chrs_set str_in - search str_in str_findPositionsOf_subStrQ IS OP Substr Str - returns list of Substr positions in Str, or an error str_findPositionsOf_subStrs IS OP Substrs Str - returns list of Substr positions in Str for EACH Substrs, or an error (23Oct2020 incomplete) str_prepadWith_chrs IS OP len chr str str_findPositionsOf_subStrs IS OP subStrs str to : str_findChrsInSet IS OP str flag_chrs_include chrs_set - search str_in str_findPositionsOf_subStrQ IS OP str substr - returns list of Substr positions in Str, or an error str_findPositionsOf_subStrL IS OP str SubstrL - returns list of Substr positions in Str for EACH Substrs, or an error (23Oct2020 incomplete) str_prepadWith_chrs IS OP str len chr str_findPositionsOf_subStr IS OP str subStr # 06Sep2021 for deletion : lead_chrs IS str_prepadWith_chr 10Sep2021 completed [optr, test]s for tester_str_find_subStr IS { vector_from_list_test ; str_find1stChrOf_subStr_test ; str_findAllChrsOf_subStr_test ; str_find1stChrOf_subStrL_test ; str_findAllChrsOf_subStrL_test ; } >> transferred to string[,- tests].ndf 10Sep2021 But where is 'str_findPositionsOf_subStr[,L]' used? +-----+ fileops.ndf >>> loading start : file_ops.ndf ?undefined identifier: PICK ( STR_SPLITBY_CHAR <***> ` RESULT ) ?undefined identifier: FLAG_BACKUP THEN PATH_BACKUPDATED_RETAIN <***> P_INN ; ENDIF ?undefined identifier: % backup first!! ; ?undefined identifier: `, EACHRIGHT STR_SPLITBY_CHAR <***> LINEL ; N_STRS ?undefined identifier: ROWTXTLIST ( STRING_CUT_BY_CHAR <***> CHR_TAB LINE ) ?undefined identifier: := FIRST STRING_SPLIT_BY_STRING <***> ' ' TEXT_LINE ; <<< loading ended : file_ops.ndf # 06Sep2021 for deletion : string_split_by_string_with_sub IS str_replaceSubTo_strList str_replaceSubTo_strList IS OP str subStrOld subStrNew str_findPositionsOf_subStr IS OP str subStr Build new optr : # 06Sep2021 initial from str_splitWith_subStr str_cutBy_subStr IS OP str subStr +-----+ windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf >>> loading start : windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf ?undefined identifier: ; ( FIND_STRING <***> 'dell64' WINSTR ) ?undefined identifier: WINTITLE_GOT := WINIDORTITLE_GET_WINTITLE <***> WINID ; ?undefined identifier: ( EACH TOSTRING <***> WINPOSN ) ( <<< loading ended : windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf ******** 24Oct2020 str_splitLftRgtTo_midlIndices_StrList - still disrupted # qnial> mailto_text := ' mailto paper formatting' mailto paper formatting qnial> indices lineList := str_splitLftRgtTo_midlIndices_StrList 'mailto:' '">' mailto_text +-+----------------------------------------------------------------------------------------------------------- |2|+---------------+-------+---------------------------------------------------------------------------------- | || |mailto paper formatting|| -----+--+----------------------------+| --------------------------------------+ # qnial> argList := ' ' '%20' EACHRIGHT append lineList#indices +------------------------------------------------------------------------------------------------------------- |+-+---+------------------------------------------------------------------------------------------------------ || |%20|IEEE WCCI 2020 HELP daemon ?subject=IEEE WCCI 2020 HELP : paper formatting&b |+-+---+------------------------------------------------------------------------------------------------------ +------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------+ -----------------------------------------------------------------------------------------------+| ody=Provide a description of your problem, including any [output from the system, error codes] || -----------------------------------------------------------------------------------------------+| ------------------------------------------------------------------------------------------------+ # qnial> line := link lineList mailto paper formatting >> Looks good! ******** 23Oct2020 str_splitLftRgtTo_midlIndices_StrList - put in fault condition # mailto_text := '"mailto:IEEE%20WCCI%202020%20HELP%20daemon%20?subject=IEEE%20WCCI%202020%20HELP%20:%20paper%20formatting&body=Provide%20a%20description%20of%20your%20problem,%20including%20any%20[output%20from%20the%20system,%20error%20codes]%20">' # str_splitLftRgtTo_midlIndices_StrList 'mailto:' '">' mailto_text >> almost, fails : ++------------------------------------------------------------------------------------------------------------ ||+----------------------------------------------------------------------------------------------------------- |||"mailto:IEEE%20WCCI%202020%20HELP%20daemon%20?subject=IEEE%20WCCI%202020%20HELP%20:%2 ||+----------------------------------------------------------------------------------------------------------- ++------------------------------------------------------------------------------------------------------------ -------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------- 0paper%20formatting&body=Provide%20a%20description%20of%20your%20problem,%20including%20any%20[output%20fr -------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------- -------------------------------------------+ --------------------------------------+--++| om%20the%20system,%20error%20codes]%20|">||| --------------------------------------+--++| -------------------------------------------+ >> maybe due to mailto near start of str? >> I re-ran string_splitWith_string_test -> all tests work great! That's probably not the problem optr >> Besides, it picks out "> but not mailto: ??? >> NUTS!!! 'maito:' should be 'mailto:' # mailto_text := 'gobacktonature "mailto:IEEE%20WCCI%202020%20HELP%20daemon%20?subject=IEEE%20WCCI%202020%20HELP%20:%20paper%20formatting&body=Provide%20a%20description%20of%20your%20problem,%20including%20any%20[output%20from%20the%20system,%20error%20codes]%20">' # I put a fault condition in : # str_splitLftRgtTo_midlIndices_StrList 'maito:' '">' mailto_text >> OK - fault returned # mailto_text := 'mailto:IEEE WCCI 2020 HELP daemon ?subject=IEEE WCCI 2020 HELP : paper formatting&body=Provide a description of your problem, including any [output from the system, error codes] ">' # a := ' ' # str_splitLftRgtTo_midlIndices_StrList '[#=; ' ' ;=#' a # old code % If n_splits = 1, headSplits would be a solitary string, so strLft is not in line ; IF (0 < (gage shape splits)) THEN ******** 23Oct2020 I removed [grep, sed] operators, as I now hybridize QNial and Unix [commands, bash] IF flag_debug THEN write 'loading grep' ; ENDIF ; #] grep IS OP grep_expr - Unix grep-like (?global, general? regular expression and parsing) # 18Jan2018 initial version # This is NOT a general form of grep, not at all!! But it is a start... grep IS OP grep_expr file_or_string { p_grep := link d_QNial_temp 'grep_temp.txt' ; host link 'rm "' p_grep '"' ; % ; fin := open file_or_string "r ; IF = fin (fault '?No such file or directory') THEN %[write, host] link 'echo "' file_or_string '" | sed ''s/$/\0/'' | tr \\0 \\n | grep -e "' grep_expr '" >>"' p_grep '" ' ; host link 'echo "' file_or_string '" | sed ''s/$/\0/'' | tr \\0 \\n | grep -e "' grep_expr '" >>"' p_grep '" ' ; ELSE %[write, host] link 'cat "' file_or_string '" | sed ''s/$/\0/'' | tr \\0 \\n | grep -e "' grep_expr '" >>"' p_grep '" ' ; host link 'cat "' file_or_string '" | sed ''s/$/\0/'' | tr \\0 \\n | grep -e "' grep_expr '" >>"' p_grep '" ' ; ENDIF ; close fin ; % ; fin := open p_grep "r ; s_out := null ; WHILE (NOT isfault (str_in := readfile fin)) DO s_out := append s_out str_in ; ENDWHILE ; close fin ; % ; s_out := (null EACHRIGHT unequal s_out) sublist s_out ; IF (= 1 (gage shape s_out)) THEN s_out := first s_out ; ENDIF ; s_out } IF flag_debug THEN write 'loading regexp_substitute' ; ENDIF ; #] regexp_substitute IS OP R S T - replace one or more substrings in string T # that match the regular expression pattern R with string S # This is a "temporary definition" until I can recompile QNial with the regular expression operators!!! # This only provides BASIC capability - not the regular expressions themselves! # derived from : str_replaceSubTo_strList IS OP S1 S2 S3 # old QNial operator - not installed with newer versions #D *regexp R S used to find the first substring in a string S that matches a regular expression pattern given by R. #D *regexp_match R S [O] used to test whether there is a substring in a string S that matches a regular expression pattern given by R # for development, see "/media/bill/SWAPPER/Qnial/MY_NDFS/strings - work.txt" # 19Jan2018 Get rid of this eventually - but keep for now for legacy programs! regexp_substitute IS OP S1 S2 S3 { S1_length S3_length := EACH tally S1 S3 ; string_block := solitary S3 ; IF S1_length < S3_length THEN FOR j WITH tell (- S3_length S1_length + 1) DO in_S1 := l ; FOR i WITH tell S1_length DO IF S1|[i] ~= S3|[j+i] THEN in_S1 := o ; EXIT null; ENDIF ; ENDFOR ; IF in_S1 = l THEN part1 part2 := [tell j, (j + S1_length) EACHRIGHT + tell (S3_length - j - S1_length)] EACHLEFT choose S3 ; result_back := regexp_substitute S1 S2 part2 ; IF isfault result_back THEN string_block := null ; ELSE string_block := (part1 (S2 link (first result_back))) link (rest result_back) ; ENDIF ; EXIT null ; % is this even useful? - YES! ; ENDIF ; ENDFOR ; ENDIF ; link string_block } #] sed code - based on Unix sed IF flag_debug THEN write 'loading sed_prep_string' ; ENDIF ; #] sed_prep_string IS OP stringer - escapes characters that will "corrupt" sed # 08Apr2018 first version - pass-through lines starting with "#" # 09Apr2018 - it is "difficult" to create test strings that don't generate errors with the interpreter # where apostrophes appear in a string, link the entire expression broke up around the apostrophes, # and replace the latter with "`'" (see examples below) # in other workds, one MUST manually contruct each string that has apostrophes!! # 10Apr2018 - not useful for sed, as I must pre-setup strings manually!! # 15Apr2018 - It is taking me TEN TIMES longer to code with sed!!! It would be best to stay in Qnial! # too many freakish character behaviours # from https://unix.stackexchange.com/questions/32907/what-characters-do-i-need-to-escape-when-using-sed-in-a-sh-script For details, see "/media/bill/SWAPPER/System_maintenance/Linux/sed notes.txt" in the list of characters matched by a bracket expression '$.*/[\]^' -> most of these are NOT legal symbol characters, so they won't be in extracted symbols In the replacement text: '&\/\n' ; # 15Apr2018 Currently I do this manually sed_prep_string IS OP stringer { LOCAL i s_len s_out ; s_len := gage shape stringer ; s_out := `' ; FOR i WITH tell s_len DO IF (= `' stringer@i) THEN s_out := s_out link `' `' ; ELSEIF (= `" stringer@i) THEN s_out := s_out link `\ `" ; ELSE s_out := s_out link stringer@i ; ENDIF ; ENDFOR ; s_out := link s_out `' ; %write_debug s_out ; s_out } IF flag_debug THEN write 'loading sed_unprep_string' ; ENDIF ; #] sed_unprep_string IS OP stringer sed_unprep_string IS OP stringer { LOCAL i s_len s_out ; str_short := (front rest) stringer ; s_len := gage shape str_short ; s_out := '' ; flag_apo := o ; FOR i WITH tell s_len DO IF ( = `' str_short@i) THEN IF flag_apo THEN flag_apo := o ; ELSE flag_apo := l ; s_out := s_out link str_short@i ; ENDIF ; ELSE flag_apo := o ; s_out := s_out link str_short@i ; ENDIF ; ENDFOR ; s_out } IF flag_debug THEN write 'loading sed' ; ENDIF ; #] sed IS OP sed_expr str - sed process string with given sed_expression # partial implementation using host calls to Unix sed (simple editor - line by line) # 19Jan2018 www.BillHowell.ca initial # sed at present only takes a simple string, not an array of strings # However - with QNial it is simple to do, for string_array consisting of strings : # string_array EACHLEFT sed sed_expr # LMDE2 sed does NOT replace with a newline! (typical of man [Unix, Linus] O/S's) # 10Apr2018 for the code development problems, see "/media/bill/SWAPPER/Qnial/MY_NDFS/strings - sed, sed_file coding development.txt" # 05May2018 modified to use flag_pre for file tranlsation sed IS OP flag_pre sed_expr str { LOCAL fin str_in p_sed s_out textnew ; NONLOCAL sym_chrs ; p_sed := link d_QNial_temp 'sed_temp.txt' ; IF flag_pre THEN textnew := str ; ELSE textnew := sed_prep_string str ; ENDIF ; %write_debug link 'textnew= ' textnew ; host link ' echo "' textnew '" | sed ' sed_expr ' >"' p_sed '" ' ; fin := open p_sed "r ; IF flag_pre THEN textnew := readfile fin ; ELSE textnew := sed_unprep_string readfile fin ; ENDIF ; close fin ; textnew } sym_chrs := 'A-Za-z0-9θφ' ; # 17Apr2018 try sym_chrs := 'A-Za-z0-9α-ω' ; # sed_pre IS OP str - pre-treats all lines in a file, then goes to "_pre" versions of other ops # the hope here is to speed things up # 05May2018 initial # from https://unix.stackexchange.com/questions/32907/what-characters-do-i-need-to-escape-when-using-sed-in-a-sh-script For details, see "/media/bill/SWAPPER/System_maintenance/Linux/sed notes.txt" in the list of characters matched by a bracket expression '$.*/[\]^' -> most of these are NOT legal symbol characters, so they won't be in extracted symbols In the replacement text: '&\/\n' ; # 15Apr2018 Currently I do this manually ******** 19Oct2020 remove [legacy, redundant] operators #] str_splitBy_spaces IS OP str - legacy, use string_split[,With]_string!! # to??words in 'niallib7/towords.ndf' towords IS str_splitBy_spaces #] towords1 IS OP str - legacy extract strings (bounded by space only!), use string_split[,With]_string!! IF flag_debug THEN write 'loading towords1' ; ENDIF ; #] towords1 IS OP Str - This operation returns the list of STRINGS (not phrases!!) # of the words in the string. # Howell: 16Feb07 based on QNial library 'towords' # # Words in the string are character sequences separated by one or more blanks. # For example, the string 'See Spot run.' has 3 words: "See", "Spot" and # run.". towords1 IS OP Str { Strings := EACH pass ( ` match Str cut Str ) } # Examples: w := 'See sam cok the dinner ' ; Result +---+---+---+---+------+ |See|sam|cok|the|dinner| +---+---+---+---+------+ Replace this with str_splitLftRgtTo_indices_StrList str_replaceIn_strList : IF flag_debug THEN write 'loading strList_replaceMidl' ; ENDIF ; #] strList_replaceMidl IS OP midlOld midlNew strLft strRgt strList - replace a [left, right]-end-bracketed middleString in str, the midlStr is constant # focus is on extracting subMid # 05Oct2020 initial, 06Oct2020 created string_splitWith_string to do it better # strRgt must be 2 positions from strLft or that st_head is not valid ; # in real life, some strRgt will occur in the next split, without a "middle" (null) # this does not account for nesting of [strLft, strRgt] # 06Oct2020 essentially assumes every strLft has only one strRgt # HREF - need to replace HREF(strLft) as SRC(strRgt) is in HREF(strLft) # more general approach - apply str_replaceIn_strList, so I can have a null operation # ignore for now, but look for examples later # 08Oct2020 for website, currently I only do [HREF, SRC] sub # in future, [midlOld, midlNew] should be lists of strings for more power # this operator may be applied many times for different [midlOld midlNew strLft strRgt strHdSub] # so only link all subStrings AFTER doing all # str MUST be a solitary is a single str is input! # none of the markers can be a subset of another marker that is subsequently used in string_splitWith_string # up to the application of strLftTail_replace_midl when a wise choice of new # [strLft, strRgt]s will avoid the problem IF flag_break THEN BREAK ; ENDIF ; strList_replaceMidl IS OP midlOld midlNew strLft strRgt strList { LOCAL i i_lfts i_rgts indices splits valids ; splits := strList ; indices := tell (gage shape splits) ; i_lfts := (strLft EACHRIGHT = splits) sublist indices ; i_rgts := (strRgt EACHRIGHT = splits) sublist indices ; valids := 1 + (((i_lfts + 2) EACHLEFT in i_rgts) sublist i_lfts) ; FOR i WITH valids DO splits@i := str_replace_subStr midlOld midlNew splits@i ; ENDFOR ; splits } sub_head_midl_tail IS strList_replaceMidl # test # '
  • ' '' '7,500 years of history - This is the same challenge' # strList_replaceMidl '%20' ' ' '||' '||' ('
  • ' '||' 'http://www.billhowell.ca/Civilisations%20and%20sun/Howell%20-%20radioisotopes%20and%20history.jpg' '||' '7,500 years of history - This is the same challenge') +-----+--------+-------------------------------------------------------------------------------------+-------- |
  • ||||http://www.billhowell.ca/Civilisations and sun/Howell - radioisotopes and history.jpg||| +-----+--------+-------------------------------------------------------------------------------------+-------- +-------------------------------------------------------+ |7,500 years of history - This is the same challenge| +-------------------------------------------------------+ >> OK # old code strHeadTail_replace_midl IS OP midlOld midlNew strHead strTail split_inn tell_Lfts := tell (gage shape i_lfts) ; splits := strLft ??str_splitBy_str strList ; # strings_between '[#=; ' ' ;=#]' ' Neural Nets ' ******** 06Oct2020 #] string_splitWith_string IS OP S1 S2 - splits S2 at each point that S1 occurs, S1 separate taken from string_split_by_string I have not (yet) developed a braoder test operator in "strings- tests.ndf" simple testing - see "strings.ndf" # ************************** # find_string IS OPERATION Substr Str - returns list of Substr positions in Str, or an error # from niallib File: findstri.ndf # BUT MODIFIED - to return the starting position(s) of substrings! # original version : # Findstring is an operation that compares two strings to see if one is contained in the other. Substr is a string. If Substr is in Str, True is returned. (CHANGED - returns EITHER the starting position, OR (gage shape Str)). find_string IS OPERATION Substr Str { Position := first Substr findall Str ; Substr in ( tally Substr EACHRIGHT take ( Position EACHLEFT drop Str ) ) } # 130509 www.BillHowell.ca version : # Findstring is an operation that compares two strings to see if one is # contained in the other. Substr is a string. # If Substr is in Str, then : # - the (multiple) positions of Substr are returned, # or nul, if there is no occurence of substr %write 'loadding find_string' ; find_string IS OPERATION Substr Str { Position := first Substr findall Str ; (Substr EACHRIGHT = ( tally Substr EACHRIGHT take ( Position EACHLEFT drop Str ) ) ) sublist position } # Examples: # 'the' EACHRIGHT find_string ['There is a way','and you have the','it'] #++--++ #||13|| #++--++ # find_string '">' '
  • Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning Zhen He, Shaobing Gao, Liang Xiao, Daxue Liu, Hangen He, David Barber
  • ' #>> Result : 129 278 364 453 540 626 714 # find_string 'https://papers.nips.cc/paper/' '
  • Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning Zhen He, Shaobing Gao, Liang Xiao, Daxue Liu, Hangen He, David Barber
  • ' #>> Result in diagram mode : #+--+ #|21| #+--+ # find_string '<|p' '
  • Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning Zhen He, Shaobing Gao, Liang Xiao, Daxue Liu, Hangen He, David Barber
  • ' #>> Result in diagram mode : #+ #| #+ # find_string 'nips' '
  • Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning Zhen He, Shaobing Gao, Liang Xiao, Daxue Liu, Hangen He, David Barber
  • ' #>> Result in diagram mode : #+--+---+---+---+---+---+---+ #|36|236|317|408|495|581|667| #+--+---+---+---+---+---+---+ # find_string null '
  • Wider and Deeper, Cheaper and Faster: Tensorized LSTMs for Sequence Learning Zhen He, Shaobing Gao, Liang Xiao, Daxue Liu, Hangen He, David Barber
  • ' #>> Result in diagram mode : # nothing (null?) # Find_strings is an operation that compares a group of strings to see if any of them is contained in another string. Substrs is a list of strings. If any string in Substrs is in Str, True is returned. find_strings IS OPERATION Substrs Str { Position := EACH first Substrs EACHLEFT findall Str ; or ( Substrs EACHBOTH in ( EACH tally Substrs EACHBOTH EACHRIGHT take ( Position EACHLEFT EACHLEFT drop Str ) ) ) } # Examples: # ['the','and'] EACHRIGHT find_strings ['There is a way','and you #have','it'] #olo # Howell's result (find_strings not modified yet) : 'the' EACHRIGHT find_string ['There is a way','and you have','it'] # 'the' find_string 'Is that their cat?' #l # *************************** # string_split_by_string_with_sub S1 S2 S3 splits S3 at each point that S1 occurs, # and puts in S2 in place of S1 # returns a list of substrings Howell: from tostring above loaddefs 'C:\Documents and Settings\William Neil Howell\My Documents\Qnial\MY_NDFS\strings.ndf' string_split_by_string_with_sub IS OP S1 S2 S3 { S1_length S3_length := EACH tally S1 S3 ; string_block := solitary S3 ; IF S1_length < S3_length THEN FOR j WITH tell (- S3_length S1_length + 1) DO in_S1 := l ; FOR i WITH tell S1_length DO IF S1|[i] ~= S3|[j+i] THEN in_S1 := o ; EXIT null; ENDIF ; ENDFOR ; IF in_S1 = l THEN part1 part2 := [tell j, (j + S1_length) EACHRIGHT + tell (S3_length - j - S1_length)] EACHLEFT choose S3 ; result_back := string_split_by_string_with_sub S1 S2 part2 ; IF isfault result_back THEN string_block := null ; ELSE string_block := (part1 (S2 link (first result_back))) link (rest result_back) ; ENDIF ; EXIT null ; % is this even useful? - YES! ; ENDIF ; ENDFOR ; ENDIF ; string_block } # old version - coudn't handle_end_of_string properly string_split_by_string_with_sub IS OP S1 S2 S3 { S1_length S3_length := EACH tally S1 S3 ; string_block := solitary S3 ; IF S1_length < S3_length THEN for_i_list := tell S1_length ; FOR j WITH tell (- S3_length S1_length) DO in_S1 := l ; FOR i WITH for_i_list DO IF S1|i ~= S3|(j+i) THEN in_S1 := o ; EXIT null; ENDIF ; ENDFOR ; IF in_S1 = l THEN part1 part2 := [tell j, (j + S1_length) EACHRIGHT + tell (S3_length - j - S1_length)] EACHLEFT choose S3 ; result_back := string_split_by_string_with_sub S1 S2 part2 ; IF isfault result_back THEN string_block := null ; ELSE string_block := (part1 (S2 link (first result_back))) link (rest result_back) ; ENDIF ; EXIT null ; % is this even useful? - YES! ; ENDIF ; ENDFOR ; ENDIF ; string_block } # Examples string_split_by_string_with_sub 'def' ' sub ' 'abcdefghij' string_split_by_string_with_sub '/*' ' sub ' 'abcd /* sdgb' string_split_by_string_with_sub '/*' ' sub ' 'abcd /* sdgb */ /*dkrifggkgfg*/' string_split_by_string_with_sub '/*' ' sub ' 'abcd /* sdgb /*jhkjk *** */ sd/*zfes /*fr' %write 'loading strings_between' ; # strings_between IS OP strStrt strEndr strIn - extracts all sub-strings in StrIn bracketed by [strStrt,strEndr] # This does NOT assume that all [strStrt,strEndr] are "partnered". # 12Feb2018 this is not an efficient routine! #******************************* # strings_between IS OP strStrt strEndr strIn - extracts all sub-strings in StrIn # bracketed by [strStrt,strEndr] strings_between IS OP strStrt strEndr strIn { shape_strStrt := gage shape strStrt ; host link 'echo >"/media/bill/ramdisk/strings_between strIn.txt" "' strIn '" ' ; fronts rears := strStrt strEndr EACHLEFT find_string strIn ; %write link link 'strings_between: fronts=' (EACH string fronts) ' rears=' (link link ((EACH string rears) EACHLEFT append ' ')) ; betweens := null ; flag_continue := l ; %write 'strings_between - before 1st FOR ' ; FOR i WITH fronts DO FOR j WITH rears DO %write i j ; IF j > i THEN betweens := betweens append (i j) ; rears := rest rears ; %flag_continue := o ; EXIT null ; ENDIF ; ENDFOR ; ENDFOR ; %write 'betweens = ' betweens ; between_strs := null ; %write 'strings_between - before 2nd FOR ' ; FOR i WITH betweens DO between_first := first i + shape_strStrt ; between_last := second i - between_first ; %write i between_first between_last ; between_strs := between_strs append ((between_first + tell between_last) choose strIn) ; ENDFOR ; %write 'leaving strings_between : between_strs' ; between_strs } %write 'loading strings_between commentary' ; # 13Feb2018 I ran tests on find_string - seems to work OK? Try situations where strStrt NOT in strIn, and other issues # No problems!?!? # I put in strings_between : write 'strings_between - strIn= ' strIn ; # Just before Segmentation fault - no problem with strIn # Change to : write link 'strings_between - strStrt strEndr = "' (first strStrt) '" "' (first strEndr) '"' ; # got to : 6876-alternating-estimation-for-structured-high-dimensional-multi-response-models # Change to : write link 'strings_between - strStrt strEndr = "' strStrt '" "' strEndr '"' ; # got to : 7152-imagination-augmented-agents-for-deep-reinforcement-learning # >> pretty good, not complete, but why??? # strStrt strEndr are NOT the problem # Change to : host link 'echo >"/media/bill/ramdisk/strings_between strIn.txt" "' dtrIn '" ' ; # WOW!!! looking at "/media/bill/ramdisk/strings_between strIn.txt" #
  • šžþÿref=https://papers.nips.cc/paper/6797-stabilizing-training-of-generative-adversarial-networks-through-regularization>Stabilizing Training of Generative Adversarial Networks through Regularization # notice strange characters "
  • šžþÿref=" # BUT - this is not in original file! : #
  • Stabilizing Training of Generative Adversarial Networks through Regularization # Try again, looking at "/media/bill/ramdisk/strings_between strIn.txt" #
  • šžþÿref=https://papers.nips.cc/paper/6797-stabilizing-training-of-generative-adversarial-networks # EACH charrep 'šžþÿ' #-59 -95 -62 -98 -61 -66 -61 -65 # Is this due to excessive string length? # a := '
  • Stabilizing Training of Generative Adversarial Networks through Regularization Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, Thomas Hofmann
  • ' # gage shape a #600 # Pre-filter file to get rid of non-ASCII? # use "/media/bill/PROJECTS/Qnial/MY_NDFS/email - extract, sort, cull addresses from text.ndf" # unicode_to_ascii IS OP fin_name fas_name #{ host (link 'iconv -f utf-8 -t ascii -c "' fin_name '" > "' fas_name '" ') ; #} # Try to get a different failure point : change # lq_confPapers # extract_paperURLs # test strings_between # a := '<|p' # b := '|>' # c := '<|p " " ** forget it Mac *** |> "Just a joke" <|p "no way, Jose |> |> <|p big jump <|p ' # strings_between a b c # *************************** # regexp_substitute_test regexp_substitute 'def' 'XX' 'abcdefghi' abcXXghi regexp_substitute 'xyz' '456' 'abcdefghi' abcdefghi regexp_substitute 'DEF' 'YY' 'abcdefghi' "i abcYYghi regexp_substitute 'a' '3' 'all able apes pay attention' "g 3ll 3ble 3pes p3y 3ttention regexp_substitute 'A' '3' 'all able apes pay attention' "ig 3ll 3ble 3pes p3y 3ttention #********************************* # sed_prep_string development # sed_illegals ' ''04_16rev4Lucas4-15'' ' # sed_prep_string ' ( 'E,B for symmetry point charge @v_const ' ' # sed_prep_string '#------> (4-16) 'E,B for symmetry point charge @v_const ' ' ?undefined identifier: SED_PREP_STRING '#------> (4-16) ' E <***> , B FOR # sed_prep_string link '#------> (4-16) ' `' 'E,B for symmetry point charge @v_const ' `' #********************************* # sed development # best is to pre-process text by replacing text_key with newlines, for example : # I forget what this was for : tr ',' '\n' | # QNial strings : sed ''s/$/\0/'' | tr \\0 \\n | # 19Jan2018 I switched arguments to "sed IS OP sed_expr string_in " # nyet! -> 08Apr2018 string_in should be quoted!! # 09Apr2018 prep [strings, files] so that they are sed-compatible!! # 09Apr2018 sed ASSUMES that a [string, file] has been rendered sed-compatible!! # string input : use ??? # text file input : use ??? # 09Apr2018 see "translate_string" in "/media/bill/PROJECTS/Qnial/MY_NDFS/translate symbols.ndf" # to use a list of symbols and their replacements (dictionary) to translate a string # old code textnew := sed_prep_string string_in ; %[write, host] link ' echo "' textnew '" | sed >"' p_sed '" ' sed_expr ; host link ' echo "' textnew '" | sed ' sed_expr ' >"' p_sed '" ' ; sed_test IS { LOCAL sed_text sed_expr ; sed_text := '- has the term '"'(ros*cosOo - vos*t)/|ro - vo*t|'"' in the first term in parenthesis. ' ; sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; sed sed_expr sed_text } # old tests with tester # 09Apr2018 after many headache fixes : sed_text := ' - dp[dt : EIpds(POIo,t)]/Vons(particle)/cos(AOpc(POIo,t)) ' ; sed_expr := link '"s/\([^' sym_chars ']\)\(AOpc(POIo,t)\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; - dp[dt : EIpds(POIo,t)]/Vons(particle)/cos(cos(Aθoc(POIo))) >> 09Apr2018 OK # sed_text := '- I have NO real idea of why the (ros*cosOo - vos*t) pops up anyways, and why the cos term all of a sudden (spherical coords does not explain this!)' ; sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; - I have NO real idea of why the (ros*cos(Aθoc(POIo)) - vos*t) pops up anyways, and why the cos term all of a sudden (spherical coords does not explain this!) >> 09Apr2018 OK # early code sed_prep_string IF (= char_quotation stringer@i) THEN stringer_new := link stringer_new char_apostrophe ; ENDIF ; IF (in stringer@i sed_illegals) THEN s_out := s_out link `\ ; ENDIF ; # to handle quotes IF (= `" stringer@i) THEN s_out := s_out link `\ ; ENDIF ; # sed_text := '- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' ; sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; sh: 1: Syntax error: "(" unexpected - dp[dt : EIpds(POIo,t)]/Vons(particle)/cos(cos(Aθoc(POIo))) >> 10Apr2018 OOPS! returns old result? I defined LOCALS >> try double quotes with simple expressions qnial> '- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' '- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' qnial> '- has the term ""(ros*cosOo - vos*t)/|ro - vo*t|"" in the first term in parenthesis. ' - has the term ""(ros*cosOo - vos*t)/|ro - vo*t|"" in the first term in parenthesis. >> sed_prep_string add : IF (= `" stringer@i) THEN s_out := s_out link `" ; ENDIF ; sh: 1: Syntax error: "(" unexpected - dp[dt : EIpds(POIo,t)]/Vons(particle)/cos(cos(Aθoc(POIo))) >> NYET - why does OLD s_out come out? -> defined in global environment? NO - undefined. good. >> also sed_prep_string not currently in sed! >> I must "rm p_sed" sh: 1: Syntax error: "(" unexpected >> then freezes and I Ctrl-C to abort qnial qnial> link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' "s/\([^A-Za-z0-9θφ]\)\(cosOo\)\([^A-Za-z0-9θφ]\)/\1cos(Aθoc(POIo))\3/g" >> OK, that is alright >> so why does sed hang up? ??eof for empty file? >> sed add to trace : write link 's_out = ' s_out ; qnial> test_sed rm: cannot remove ‘/media/bill/RaspPi_ext4_32Gb/temp/sed_temp.txt’: No such file or directory sh: 1: Syntax error: "(" unexpected >> so changes didn't help >> sed add : textnew := sed_prep_string string_in ; qnial> test_sed rm: cannot remove ‘/media/bill/RaspPi_ext4_32Gb/temp/sed_temp.txt’: No such file or directory '- has the term ""\(ros\*cosOo - vos\*t\)\/|ro - vo\*t|"" in the first term in parenthesis. ' +-+-+-+-+-+-+-+-+--------------------------------------------------------------------------------------------- |s|_|o|u|t| |=| |'- has the term \(ros\*cos(Aθoc(POIo)) - vos\*t\)\/|ro - vo\*t| in the first term in parenth +-+-+-+-+-+-+-+-+--------------------------------------------------------------------------------------------- --------+ esis. '| --------+ '- has the term \(ros\*cos(Aθoc(POIo)) - vos\*t\)\/|ro - vo\*t| in the first term in parenthesis. ' >> OK, this is better! still have problem with disappearing quotes >> Don't need to escape sed_illegals!!! remove from sed : IF (in stringer@i sed_illegals) THEN s_out := s_out link `\ ; ENDIF ; qnial> sed_test (1st expression from sed_prep_string) '- has the term ""(ros*cosOo - vos*t)/|ro - vo*t|"" in the first term in parenthesis. ' s_out = '- has the term (ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t| in the first term in parenthesis. ' '- has the term (ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t| in the first term in parenthesis. ' >> OK, closer, but still lose quotes # sed_text := '- has the term '(ros*cosOo - vos*t)/|ro - vo*t|' in the first term in parenthesis. ' ; sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; qnial> sed_test (1st expression from sed_prep_string) ?undefined identifier: '- has the term ' ( ROS <***> * COSOO - errors found: 1 >> Can't even loaddefs !! # sed_text := '- has the term ''(ros*cosOo - vos*t)/|ro - vo*t|'' in the first term in parenthesis. ' ; sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; qnial> sed_test (1st expression from sed_prep_string) '- has the term ''(ros*cosOo - vos*t)/|ro - vo*t|'' in the first term in parenthesis. ' s_out = '- has the term ''(ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t|'' in the first term in parenthesis. ' '- has the term ''(ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t|'' in the first term in parenthesis. ' # sed_text := '- has the term ""(ros*cosOo - vos*t)/|ro - vo*t|"" in the first term in parenthesis. ' ; sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; qnial> sed_test (1st expression from sed_prep_string) '- has the term """"(ros*cosOo - vos*t)/|ro - vo*t|"""" in the first term in parenthesis. ' '- has the term (ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t| in the first term in parenthesis. ' >> sed_prep_string isn't helping - must manually prepare strings!! >> so how are quotes handled? maybe put them between apostrophes? #+-----+ # Those below use : sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; # sed_text := link '- has the term ' `" '(ros*cosOo - vos*t)/|ro - vo*t|' `" ' in the first term in parenthesis. ' ; qnial> sed_test ?undefined identifier: "' ( ROS <***> * COSOO - errors found: 1 >> maybe start&end with quote? # sed_text := "- has the term ""(ros*cosOo - vos*t)/|ro - vo*t|"" in the first term in parenthesis. " ; qnial> sed_test ?undefined identifier: := "- HAS <***> THE TERM "" errors found: 1 >> OOPS - must use QNial apos to encase strings!! # sed_text := link '- has the term '"'(ros*cosOo - vos*t)/|ro - vo*t|'"' in the first term in parenthesis. ' ; qnial> sed_test - has the term (ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t| in the first term in parenthesis. >> As in the past, I lose the quotes! damned! # sed_text := link '- has the term ' `" '(ros*cosOo - vos*t)/|ro - vo*t|' `" ' in the first term in parenthesis. ' ; qnial> sed_test - has the term (ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t| in the first term in parenthesis. >> Nope, still lose quote # sed_text := link '- has the term '"'(ros*cosOo - vos*t)/|ro - vo*t|'"' in the first term in parenthesis. ' ; qnial> sed_test (1st expression from write string_in) ?undefined identifier: "' ( ROS <***> * COSOO - errors found: 1 # sed_text := link '- has the term ' '"' '(ros*cosOo - vos*t)/|ro - vo*t|' '"' ' in the first term in parenthesis. ' ; qnial> sed_test (1st expression from write string_in) string_in= - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. sh: 1: Syntax error: "(" unexpected >> string_in is OK, but still lose expression # sed_text := link '- has the term """(ros*cosOo - vos*t)/|ro - vo*t|""" in the first term in parenthesis. ' ; qnial> sed_test (1st expression from write string_in) string_in= - has the term """(ros*cosOo - vos*t)/|ro - vo*t|""" in the first term in parenthesis. sh: 1: Syntax error: "(" unexpected ^C >> problem again, had to exit qnial # sed_text := link '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' ; qnial> sed_test (1st expression from write string_in) string_in= - has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. - has the term "(ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t|" in the first term in parenthesis. >> OK, it worked >> Now, re-introduce sed_prep_string, escaping the quotes textnew := sed_prep_string stringer ; # sed_text := link '- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' ; qnial> sed_test (1st expression from write string_in) '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' string_in= - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. sh: 1: Syntax error: "(" unexpected ^C >> quotes not a problem, now its the parenthesis error return, but I forgotr to use textnew # # same sed_text : sed_text := link '- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' ; qnial> sed_test (1st 2 expressions from write [string_in,textnew]) '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' string_in= - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. textnew= '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' '- has the term "(ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' >> GREAT!!! finally ... # old code >> replace host link 'rm "' p_sed '" ' ; s_out := null ; WHILE (~= ??eof (line_in := readfile fin)) DO s_out := link s_out (solitary line_in) ; ENDWHILE ; IF (~= ??eof s_out) THEN s_out := (null EACHRIGHT unequal s_out) sublist s_out ; IF (= 1 (gage shape s_out)) THEN s_out := first s_out ; ENDIF ; ENDIF ; >> with s_out := readfile fin ; >> check qnial> sed_test rm: cannot remove ‘/media/bill/RaspPi_ext4_32Gb/temp/sed_temp.txt’: No such file or directory string_in= - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. textnew= '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' '- has the term "(ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' >> OK works, just remove the "rm", and the comments '- has the term "(ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' >> perfect... qnial> loaddefs '/media/bill/PROJECTS/Qnial/code tests/translate symbols- tests.ndf' /media/bill/PROJECTS/Qnial/code tests/translate symbols- tests.ndf Sun Apr 15 10:56:47 2018 textnew start = - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. sed_expr= "s/\([^A-Za-z0-9θφ]\)\(d(dt :\)\([^A-Za-z0-9θφ]\)/\1d[dt:\3/g" 0 : - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. sed_expr= "s/\([^A-Za-z0-9θφ]\)\(dp[dt:\)\([^A-Za-z0-9θφ]\)/\1∂[∂t:\3/g" sed: -e expression #1, char 68: invalid reference \3 on `s' command's RHS 1 : sed_expr= "s/\([^A-Za-z0-9θφ]\)\(dp(dt:\)\([^A-Za-z0-9θφ]\)/\1∂[∂t:\3/g" 2 : >> Now there's a problem with characters - but I hadn't seen this with sed? 15Apr2018 sed was used to debug problems with [ etc in translate_string see "" #**************************** # sed_file IS OP sed_expr pname_inn pname_prep pname_out # old tests with test_sed_file sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; sed_file sed_expr sed_text >> 09Apr2018 problems with quotes '"' >> This was fixed using sed (short & easier to learn) # 10Apr2018 qnial> sed_file_test '''/media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt''' string_in= '/media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt' textnew= '''/media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt''' '''''' string_in= '' textnew= '''''' '''- has too many \\"c\\"''''s''' string_in= '- has too many \"c\"''s' textnew= '''- has too many \\"c\\"''''s''' '''- has the term \\"(ros*cosOo - vos*t)/|ro - vo*t|\\" in the first term in parenthesis. ''' string_in= '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' textnew= '''- has the term \\"(ros*cosOo - vos*t)/|ro - vo*t|\\" in the first term in parenthesis. ''' sh: 1: Syntax error: "(" unexpected ^C >> had to exit qnial >> what is doing the writes? -> sed !! >> try escaping the quotes in string_in, test with sed_prep_file first (won't work with sed?) >> change sed_prep_file : IF (= `' line_in@i) THEN s_out := s_out link `' ; ENDIF ; >> to IF (= `' line_in@i) THEN s_out := s_out link `\ ; ENDIF ; qnial> sed_file_test rm: cannot remove ‘/media/bill/RaspPi_ext4_32Gb/temp/sed_temp.txt’: No such file or directory string_in= '/media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt' textnew= '''/media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt''' string_in= '' textnew= '''''' string_in= '- has too many \"c\"\'s' textnew= '''- has too many \\"c\\"\''s''' string_in= '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' textnew= '''- has the term \\"(ros*cosOo - vos*t)/|ro - vo*t|\\" in the first term in parenthesis. ''' sh: 1: Syntax error: "(" unexpected ^C >> same problem, OOPS - sed does corrections! >> so don't call sed_prep_file? but I have to to be able to pass string? # 10Apr2018 sed_expr := link '"s/\([^' sym_chars ']\)\(cosOo\)\([^' sym_chars ']\)/\1cos(Aθoc(POIo))\3/g"' ; qnial> sed_file_test rm: cannot remove ‘/media/bill/RaspPi_ext4_32Gb/temp/sed_temp.txt’: No such file or directory string_in= '/media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt' textnew= '''/media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt''' string_in= '' textnew= '''''' string_in= '- has too many \"c\"\'s' textnew= '''- has too many \\"c\\"\''s''' string_in= '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' textnew= '''- has the term \\"(ros*cosOo - vos*t)/|ro - vo*t|\\" in the first term in parenthesis. ''' sh: 1: Syntax error: "(" unexpected ^C >> same problem >> sed_prep_string is effectively called twice - once to make the prep file, then by sed >> in sed_file, comment out : %sed_prep_file pname_inn pname_prep ; >> change : fin := open pname_prep "r ; >> to fin := open pname_inn "r ; qnial> sed_file_test ^C >> no output at all - need tracing writes, add for line_in & textnew qnial> sed_file_test 0a /media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt 0b /media/bill/PROJECTS/Lucas - Universal Force/translate inn.txt 1a 1b 2a - has too many "c"'s 2b - has too many "c"''s 3a - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. 3b - has the term "(ros*cos(Aθoc(POIo)) - vos*t)/|ro - vo*t|" in the first term in parenthesis. 4a - I have NO real idea of why the (ros*cosOo - vos*t) pops up anyways, and why the cos term all of a sudden (spherical coords does not explain this!) 4b - I have NO real idea of why the (ros*cos(Aθoc(POIo)) - vos*t) pops up anyways, and why the cos term all of a sudden (spherical coords does not explain this!) 5a 5b 6a ( dp[dt : Bi(ro - vo*t,t)] 6b ( dp[dt : Bi(ro - vo*t,t)] 7a = vos/c*Rocs(POIo)*sinOo*Pph* 7b = vos/c*Rocs(POIo)*sinOo*Pph* >> OK seems to work, except it doubles apos? #************************ 14Apr2018 - remove this from strings.ndf as it just complicates things! # sed_prep_file IS OP pname_inn pname_prep - sed process text file with constant sed_expression # 20Jan2018 www.BillHowell.ca initial # 10Apr2018 no longer used # 14Apr2018 reinstate as this is used by "translate symbols.ndf" # 14Apr2018 comment out - do string-by-string sed_prep_file IS OP pname_inn pname_prep { LOCAL fin fot i s_out s_len str_in stringer_new ; % ; fin := open pname_inn "r ; fot := open pname_prep "w ; WHILE (??eof <> (str_in := readfile fin)) DO s_out := string `' ; s_len := gage shape str_in ; FOR i WITH tell s_len DO IF (= `' str_in@i) THEN s_out := s_out link `' ; ENDIF ; IF (= `" str_in@i) THEN s_out := s_out link `\ ; ENDIF ; s_out := s_out link str_in@i ; ENDFOR ; s_out := s_out link `' ; writefile fot s_out ; ENDWHILE ; EACH close fin fot ; } # old code IF (in sed_illegals str_in@i) THEN s_out := s_out link `\ ; ENDIF ; # 10Aug2018 added : IF (= `" str_in@i) THEN s_out := s_out link `\ ; ENDIF ; # 14Apr2018 just before cleaning up : sed_file IS OP sed_expr pname_inn pname_out { LOCAL fin fot i str_in textnew ; fin := open pname_inn "r ; fot := open pname_out "w ; i := 0 ; WHILE (~= ??eof (str_in := readfile fin)) DO %write_debug link 'raw : ' str_in ; %str_in := sed_prep_string str_in ; %write_debug link (string i) 'a ' str_in ; textnew := (front rest) sed sed_expr str_in ; %write_debug link (string i) 'b ' textnew ; writefile fot textnew ; i := i + 1 ; ENDWHILE ; EACH close fin fot ; } test_comment IS OP t_name t_input t_standard t_result { LOCAL i s_len s_out t_result_trimmed ; NONLOCAL flag_test_comment ; t_result_trimmed := t_result ; IF (= t_result_trimmed t_standard) THEN IF flag_test_comment THEN EACH write (link t_name ' : OK - result matches standard') 't_input= ' t_input 't_standard, t_result_trimmed = ' t_standard t_result ; ENDIF ; ELSE EACH write (link t_name ' : FAILED - result does NOT match standard') 't_input= ' t_input 't_standard, t_result = ' t_standard t_result_trimmed ; ENDIF ; } # s_len := gage shape t_result_trimmed ; t_result_trimmed := t_result ; s_len := gage shape t_result_trimmed ; flag_apo := o ; FOR i WITH tell s_len DO IF ( = `' t_result_trimmed@i) THEN IF flag_apo THEN flag_apo := o ; ELSE flag_apo := l ; s_out := s_out link t_result_trimmed@i ; ENDIF ; ELSE s_out := s_out link t_result_trimmed@i ; ENDIF ; ENDFOR ; IF (= t_result_trimmed t_standard) THEN IF flag_test_comment THEN EACH write (link t_name ' : OK - result matches standard') 't_input= ' t_input 't_standard, t_result_trimmed = ' t_standard t_result ; ENDIF ; ELSE EACH write (link t_name ' : FAILED - result does NOT match standard') 't_input= ' t_input 't_standard, t_result = ' t_standard t_result_trimmed ; ENDIF ; # FOR i WITH tell s_len DO IF ( = `' t_result_trimmed@i) THEN IF flag_apo THEN flag_apo := o ; ELSE flag_apo := l ; s_out := s_out link t_result_trimmed@i ; ENDIF ; ELSE s_out := s_out link t_result_trimmed@i ; ENDIF ; ENDFOR ; #********************************** # sed_pre IS OP str - sed version that pre-treats all lines in a file, then goes to "_pre" versions of other ops # the hope here is to speed things up # 05May2018 initial # $ echo '- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. ' | sed "s/\'/\'\'/g" # $ echo '- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. ' | sed "s/\'/\'\'/g" # cmd := link 'echo ''- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. '' | sed ''s/\''/\''\''/g'' ' # cmd := link 'echo ''- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. '' | sed ''s/\''/\''\''/g'' ' # cmd := link 'echo ''- has the term \"(ros*cosOo - vos*t)/|ro - vo*t|\" in the first term in parenthesis. '' | sed "s/\''/\''\''/g" ' sed_file_pre_testSimple IS { LOCAL cmd d_Lucas pname_inn pname_pre ; d_Lucas := '/media/bill/PROJECTS/Lucas - Universal Force/' ; pname_src := link d_Lucas 'Howell - math of Lucas Universal Force.txt' ; pname_inn := link d_Qnial_temp 'sed_file_pre input.txt' ; pname_pre := link d_Qnial_temp 'sed_file_pre pretreated.txt' ; } # old code | sed ''s/\''\''/\''/g'' # 05May2015 Screw this "poison" between QNial and sed !!!! Just create a bash script and run it from QNial d_Qnial_mine 'strings sed_file_pre.sh' +---+ remove as this is no longer appropriate # sed_file IS OP sed_expr pname - applies the SAME sed_expr to entire file to get output file # 09Apr2018 see "translate_file" in "/media/bill/PROJECTS/Qnial/MY_NDFS/translate symbols.ndf" # to use a list of symbols and their replacements (dictionary) to translate a text file sed_file IS OP sed_expr pname_inn pname_out { LOCAL fin fot i str_in textnew ; fin := open pname_inn "r ; fot := open pname_out "w ; i := 0 ; WHILE (~= ??eof (str_in := readfile fin)) DO textnew := sed sed_expr str_in ; writefile fot textnew ; %write_debug link (string i) ' ' textnew ; i := i + 1 ; ENDWHILE ; EACH close fin fot ; } # textnew := (front rest) sed sed_expr str_in ; +---+ # remove this also # sed_pre IS OP sed_expr str - sed process string with given sed_expression # partial implementation using host calls to Unix sed (simple editor - line by line) # www.BillHowell.ca 05May2018 initial sed_string_pre IS OP sed_expr str { LOCAL fin str_in p_sed s_out textnew ; NONLOCAL sym_chrs ; p_sed := link d_QNial_temp 'sed_temp.txt' ; textnew := str ; %write_debug link 'textnew= ' textnew ; host link ' echo "' textnew '" | sed ' sed_expr ' >"' p_sed '" ' ; fin := open p_sed "r ; textnew := sed_unprep_string readfile fin ; close fin ; textnew } +---+ # remove this also # string_in_strList IS OP Substr Str_list IS OP subStr strList - check each string in str_list for subStr # returns boolean, without giving position of subStr in any of the str_list # this uses Unix grep for speed with larger str (like 29 kbytes) # STUPID - just EACH string_in str_list !!! string_in_strList IS OP Substr Str_list { LOCAL f_name fot grepper ; f_name := link d_Qnial_temp 'string_in_strList.txt' ; fot := open f_name "w ; fot EACHRIGHT writefile Str_list ; close fot ; grepper := host link 'cat "' f_name '" | grep -i "' Substr '"' ; IF (= null grepper) THEN o ELSE l ENDIF } #********************************* # string_sub IS OP sub_old sub_new str - substitutes sub_new for sub_old in str # 05May2018 based on in_string string_sub IS OP sub_old sub_new str { LOCAL flag_in len_subStr len_str i_str i_subStr textnew ; len_sub_old := gage shape sub_old ; len_str := gage shape str ; flag_in i_str i_sub_old := o 0 0 ; textnew := null ; WHILE (i_str < len_str) DO WHILE (AND ((i_sub_old + i_str) < len_str) (i_sub_old < len_sub_old) (= sub_old@i_sub_old str@(i_str + i_sub_old)) ) DO i_sub_old := i_sub_old + 1 ; ENDWHILE ; %write link '[i_str,i_sub_old]= ' (string i_str) ', ' (string i_sub_old) ; IF (len_sub_old = i_sub_old) THEN textnew := link textnew sub_new ; i_str := i_str + len_sub_old - 2 ; ELSEIF (0 < i_sub_old) THEN textnew := link textnew str@(i_str - tell (i_sub_old)) ; ELSEIF (0 = i_sub_old) THEN textnew := link textnew str@i_str ; ELSE fault '?string_sub - i_sub_old is anomalous' ; ENDIF ; i_str := i_str + 1 ; i_sub_old := 0 ; ENDWHILE ; textnew } qnial> string_sub_test # string_sub_test example 1 : OK - result matches standard t_input, t_standard, t_result = +--+----+------------------------------------+ |to|toys|Hello to my new big to see something| +--+----+------------------------------------+ Hello toys my new big toys see something Hello toys my new big toys see something # string_sub_test example 2 : OK - result matches standard t_input, t_standard, t_result = +----+--------+------------------------------------+ |ew b|dinosaur|Hello to my new big to see something| +----+--------+------------------------------------+ Hello to my ndinosaurig to see something Hello to my ndinosaurig to see something # string_sub_test example 3 : OK - result matches standard t_input, t_standard, t_result = +--------+--------+------------------------------------+ |new big |dinosaur|Hello to my new big to see something| +--------+--------+------------------------------------+ Hello to my dinosaurto see something Hello to my dinosaurto see something # string_sub_test example 4 : OK - result matches standard t_input, t_standard, t_result = +-------+--------+------------------------------------+ |new big|dinosaur|Hello to my new big to see something| +-------+--------+------------------------------------+ Hello to my dinosaur to see something Hello to my dinosaur to see something # string_sub_test example 5 : OK - result matches standard t_input, t_standard, t_result = +--------+--------+------------------------------------+ |airplane|dinosaur|Hello to my new big to see something| +--------+--------+------------------------------------+ Hello to my new big to see something Hello to my new big to see something # string_sub_test example 6 : FAILED - result does NOT match standard t_input, t_standard, t_result = +---------+------------+---------------------------------------------------------------------------------+ |ro - vo*t||Rpcv(POIp)||- has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis.| +---------+------------+---------------------------------------------------------------------------------+ - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. - h a s t h e t e r m " ( ?address o s * c o s O o - v o s * t ) / R p c s ( P O I p ) " i n t h e f i r s t t e r m i n p a r e n t h e s i s . # string_sub_test example 7 : OK - result matches standard t_input, t_standard, t_result = +-----------+----------+------------------------------------------------------------------------------------+ ||ro - vo*t||Rpcs(POIp)|- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. | +-----------+----------+------------------------------------------------------------------------------------+ - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. >> example 6 is a problem! I've been trying to track it all afternoon # string_sub_test example 6 : FAILED - result does NOT match standard t_input, t_standard, t_result = +---------+------------+---------------------------------------------------------------------------------+ |ro - vo*t||Rpcv(POIp)||- has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis.| +---------+------------+---------------------------------------------------------------------------------+ - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. - h a s t h e t e r m " ( ?address o s * c o s O o - v o s * t ) / R p c s ( P O I p ) " i n t h e f i r s t t e r m i n p a r e n t h e s i s . >> The problem was my erroneous treatment of when several chars matched, but not the whole substring >> Now it looks like IF (len_sub_old = i_sub_old) THEN textnew := link textnew sub_new ; i_str := i_str + len_sub_old - 1 ; ELSEIF (0 <= i_sub_old) THEN textnew := link textnew str@i_str ; ELSE write fault '?string_sub - i_sub_old is anomalous' ; ENDIF ; qnial> string_sub_test # string_sub_test example 1 : OK - result matches standard t_input, t_standard, t_result = +--+----+------------------------------------+ |to|toys|Hello to my new big to see something| +--+----+------------------------------------+ Hello toys my new big toys see something Hello toys my new big toys see something # string_sub_test example 2 : OK - result matches standard t_input, t_standard, t_result = +----+--------+------------------------------------+ |ew b|dinosaur|Hello to my new big to see something| +----+--------+------------------------------------+ Hello to my ndinosaurig to see something Hello to my ndinosaurig to see something # string_sub_test example 3 : OK - result matches standard t_input, t_standard, t_result = +--------+--------+------------------------------------+ |new big |dinosaur|Hello to my new big to see something| +--------+--------+------------------------------------+ Hello to my dinosaurto see something Hello to my dinosaurto see something # string_sub_test example 4 : OK - result matches standard t_input, t_standard, t_result = +-------+--------+------------------------------------+ |new big|dinosaur|Hello to my new big to see something| +-------+--------+------------------------------------+ Hello to my dinosaur to see something Hello to my dinosaur to see something # string_sub_test example 5 : OK - result matches standard t_input, t_standard, t_result = +--------+--------+------------------------------------+ |airplane|dinosaur|Hello to my new big to see something| +--------+--------+------------------------------------+ Hello to my new big to see something Hello to my new big to see something # string_sub_test example 6 : OK - result matches standard t_input, t_standard, t_result = +---------+------------+---------------------------------------------------------------------------------+ |ro - vo*t||Rpcv(POIp)||- has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis.| +---------+------------+---------------------------------------------------------------------------------+ - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. # string_sub_test example 7 : OK - result matches standard t_input, t_standard, t_result = +-----------+----------+------------------------------------------------------------------------------------+ ||ro - vo*t||Rpcs(POIp)|- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. | +-----------+----------+------------------------------------------------------------------------------------+ - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. - has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. # string_sub_test example 9 : OK - result matches standard t_input, t_standard, t_result = +---+----------+-----------------------------------------------------------------------------------+ |ros|Rocs(POIo)|- has the term "(ros*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. | +---+----------+-----------------------------------------------------------------------------------+ - has the term "(Rocs(POIo)*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. - has the term "(Rocs(POIo)*cosOo - vos*t)/Rpcs(POIp)" in the first term in parenthesis. >> With this, examples [1-7,9] are fine, but qnial goes into an infinite loop with example 8 I put in a conditional to avoid processing nulls : IF (NOT OR (sub_old sub_new EACHLEFT = null)) THEN ... ELSE textnew := null ; ENDIF ; % ; IF (= null textnew) THEN str ELSE textnew ENDIF qnial> string_sub_test # string_sub_test example 8 : OK - result matches standard t_input, t_standard, t_result = +++------------------------------------------------------------------------------------+ |||- has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. | +++------------------------------------------------------------------------------------+ - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. - has the term "(ros*cosOo - vos*t)/|ro - vo*t|" in the first term in parenthesis. >> So now example #8 also works! *********************************** 26May2018 I removed chrs_before_after_str_i as it was "consumed" by chrStr_get # chrs_before_after_str_i IS OP direction i_str str - returning string representing # a [byte,unicode] char * [before,after] i_str in str # 21May2018 initial # i_str will typically be the index of the [first,last] character of a subStr in string # use POSITIVE i_str for chars AFTER end of i_str # NEGATIVE i_str for before BEFORE start of i_str # returns : nul - if at [start,end] of str; string byte-char for non-negative codes; string of one-to-three-byte unicode # most of my unicodes are 3 byte, so the first conditional should use that assumption, then [2,4,1] # but if most unicodes are 3-byte, I might as well start assuming possibly 4-byte unicode ; chrs_before_after_str_i IS OP direction i_str str { LOCAL chr_code chr_code2 chr_code3 chr_new flag_direction i len limit ; len := gage shape str ; chr_new := 'chr_unicode_before_after_str_i ERROR : flag_direction unknown' ; flag_direction := l ; IF (direction = -1) THEN n_delta := i ; ELSEIF (direction = 1) THEN n_delta := len - i ; ELSE flag_direction := o ; ENDIF ; % ; IF flag_direction THEN % ; IF (n_delta >= 4 ) THEN i := direction + i_str ; IF (= 1 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE i := direction + i ; IF (= 2 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE i := direction + i ; IF (= 3 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE i := direction + i ; IF (= 4 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE chr_new := null ; ENDIF ; ENDIF ; ENDIF ; ENDIF ; ELSEIF (n_delta = 3 ) THEN i := direction + i_str ; IF (= 1 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE i := direction + i ; IF (= 2 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE i := direction + i ; IF (= 3 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE chr_new := null ; ENDIF ; ENDIF ; ENDIF ; ELSEIF (n_delta = 2 ) THEN i := direction + i_str ; IF (= 1 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE i := direction + i ; IF (= 2 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE chr_new := null ; ENDIF ; ENDIF ; ELSEIF (n_delta = 1 ) THEN i := direction + i_str ; IF (= 1 (unicode_bytes str@i)) THEN chr_new := chrStr_get str len i ; ELSE chr_new := null ; ENDIF ; ELSE chr_new := null ; ENDIF ; jump codeList := chr_code chr_code2 chr_code3 ; ELSE codeList := chr_code chr_code2 ; ENDIF ; ELSE codeList := chr_code chr_code2 ; ENDIF ; ELSE codeList := chr_code ; ENDIF ; ELSE codeList := chr_code ; ENDIF ; ELSE codeList := chr_code ; ENDIF ; ELSE codeList := null ; ENDIF ; IF (direction = -1) THEN codeList := reverse codeList ; ENDIF ; chr_new := char codeList ; % ; ENDIF ; %(flag_direction) ; string chr_new } ***************************** 26Jun2018 change of approach to unicode!! get rid of chrStr_get just process string as chrList!!! +-----+ # chrStr_get IS OP direction i_str str - return an [ASCII, unicode] character as a string, null # - direction =[-1 ends, 1 starts] at i_str # This is used by : # strings.ndf sym_stdForm # symbols translate.ndf symExpr_sub # symbols extract.ndf symExtract_stringPosn, symExprExtract_stringPosn # ~23May2018 initial, 25May2018 added chrStrGet_direction as global argument to consolidate "chrs_before_after_str_i" # 26May2018 replace old chrs_before_after_str_i with chrStr_get # 24Jun2018 complete revamp of code chrStr_get IS OP dirn i_str str { LOCAL c_series codeSeq dirn i i_chk i_steps jump len result steps ; % ; len := gage shape str ; jump := 0 ; result := (fault '?chrStr_get ERROR: direction is incorrect') 0 ; % ; IF (OR (len <= i_str) (i_str < 0)) THEN result := (fault '?chrStr_get i_str out of range') 0 ; % ; ELSE % most characters are normal ASCII, so treat them first ; c0 := charrep str@i_str ; IF (AND ( 0 <= c0) (c0 <= 127)) THEN codeSeq := c0 ; jump := 0 ; result := (string char codeSeq) jump ; % ; ELSE % here we assume that (127 < code < 128) are NOT used for unicode chars (dangerous assumption) ; % so all remaining characters are first treated as unicode negative codes, if not they are passed on ; codeSeq := null ; IF (= -1 dirn) THEN steps := -1 * reverse steps ; ENDIF ; i_steps := i_str + (0 1 2 3) ; i_steps := (i_steps EACHLEFT >= 0) sublist i_steps ; % select only "legal" indexes ; c_series := EACH charrep str#(i_steps) ; % select only initial contiguous series of negative codes (otherwise errors if negatives split!!!) ; FOR i WITH (tell gage shape c_series) DO IF (c_series@i >= 0) THEN EXIT '' ; ENDIF ; c_series := i take c_series ; n_chars := gage shape c_series ; chrs_HFLN_2 := EACH char unicodes_HLFN_2 ; chrs_HFLN_3 := EACH char unicodes_HLFN_3 ; % ; % In Lucas Universal Force project, the most common unicode types are 3-bytes, so check them first ; chr_found := o ; IF (n_chars >= 3) THEN chr := link codeSeq c0 c1 c2 ; jump := 2 ; IF (string_in chrs_HLFN_3) % Now consider 2-byte unicodes ; IF (AND (NOT chr_found) (n_chars >= 2)) THEN codeSeq := link codeSeq c0 c1 ; jump := 1 ; ELSE %Now consider 4-byte unicodes (I don't use any at present) ; IF (AND (NOT chr_found) (n_chars >= 4)) THEN codeSeq := link codeSeq c0 c1 c2 c3 ; jump := 3 ; % I haven't run into any one-byte unicodes yet (-ve), but this is the fallback if all above fail ; ELSE c0 := charrep str#(i_steps@0) ; codeSeq := c0 ; jump := 0 ; ENDIF ; ENDIF ; ENDIF ; ENDIF ; ENDIF ; ENDIF ; result := (string char codeSeq) jump ; % ; ENDIF ; ENDIF ; result } +-----+ change : +-----+ # sym_stdForm IS OP symList - to put [symbols,expressions] in a standard format for comparison - mostly remove [spaces,tabs] # 24Apr2018 initial # 24May2018 modified to handle unicode, code taken from symExtract_string # remove all [spaces,tabs] EXCEPT ensure 1 space around each '-+', retain '.', and following each chrs_stop_paren=')]}>' sym_stdForm IS OP str { LOCAL chr i chr_code chr_code1 chr_code2 chr_code3 chr_new jump len symNew ; NONLOCAL chrs_blank ; % ; chr_new := null ; len := gage shape str ; i := 0 ; % ; WHILE (i < len) DO chr_new jump := chrStr_get 1 i str ; IF (in_string chr_new chrs_blank) THEN null ; ELSEIF (in_string chr_new '+-=<>') THEN symNew := link symNew ' ' chr_new ' ' ; ELSEIF (in_string chr_new ':' ) THEN symNew := link symNew chr_new ' ' ; ELSE symNew := link symNew chr_new ; ENDIF ; i := i + jump + 1 ; ENDWHILE ; % ; symNew } # loaddefs link d_Qnial_mine 'symbols stdForm.ndf' +-----+ to : +-----+ # sym_stdForm IS OP symList - to put [symbols,expressions] in a standard format for comparison - mostly remove [spaces,tabs] # 24Apr2018 initial # 24May2018 modified to handle unicode, code taken from symExtract_string # 26Jun2018 revamped to simply convert str to chrList # remove all [spaces,tabs] EXCEPT ensure 1 space around each '-+', retain '.', and following each chrs_stop_paren=')]}>' sym_stdForm IS OP str { LOCAL chr i chr_code chr_code1 chr_code2 chr_code3 chr_new jump len symNew ; NONLOCAL chrs_blank ; % ; chrList := EACH string str ; len := gage shape chrList ; i := 0 ; % ; WHILE (i < len) DO IF (in chrList@i chrs_blank) THEN null ; ELSEIF (in chrList@i '+-=<>') THEN symNew := link symNew ' ' chrList@i ' ' ; ELSEIF (in chrList@i ':' ) THEN symNew := link symNew chrList@i ' ' ; ELSE symNew := link symNew chrList@i ; ENDIF ; i := i + 1 ; ENDWHILE ; % ; symNew } +-----+ Now, just get rid of +-----+ # chrStr_get_test IS - chrStr_get_test IS { LOCAL t_name t_input t_standard t_result ; EACH write_testStr '#+-----+' (link 'chrStr_get_test, ' timestamp) ; % ; t_name := '# chrStr_get_test example 1 vrp' ; t_input := 1 0 'Bi(r,v,t) = q/c*(vrp)/rs^3 = (v/c)E0(r,t)' ; t_standard := 'B' 0 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link 'Notice that i_str: ' (string first t_input) ' is at the start of str (i_str = 0)' ; a := 2 pick t_input ; b := tell gage shape a ; write_testStr picture mix b a#b ; % ; t_name := '# chrStr_get_test example 1a vrp' ; t_input := 1 49 'Bi(r,v,t) = q/c*(vrp)/rs^3 = (v/c)E0(r,t)' ; t_standard := ')' 0 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link 'Notice that i_str: ' (string first t_input) ' is at the end of str, len= " (string gage shape third t_input)' ; a := 2 pick t_input ; b:= tell gage shape a ; write_testStr picture mix b a#b ; % ; t_name := '# chrStr_get_test example 1b' ; t_input := -1 42 'Bi(r,v,t) = q/c*(vrp)/rs^3 = (v/c)E0(r,t)' ; t_standard := '' 2 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link 'Notice that i_str: ' (string first t_input) ' is at the end of unicode seq, len= " (string gage shape third t_input)' ; a := 2 pick t_input ; b:= tell gage shape a ; write_testStr picture mix b a#b ; % ; t_name := '# chrStr_get_test example 2 r' ; t_input := 1 12 'e^2*(r·b)*r(rb)/|r2 - r1|^2*6*Bpe^2*Bep^2*(-3/2/π)' ; t_standard := '' 2 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link 'should be OK, reading unicode. Keep in mind that "·" adds one extra i_str!!' ; a := 2 pick t_input ; b:= tell gage shape a ; write_testStr picture mix b a#b ; % ; t_name := '# chrStr_get_test example 2a r' ; t_input := -1 19 'e^2*(r·b)*r(rb)/|r2 - r1|^2*6*Bpe^2*Bep^2*(-3/2/π)' ; t_standard := '' 2 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link 'Note that it starts at end a unicode "" (-18 -124 -87), i_str = 19' ; a := 2 pick t_input ; b:= tell gage shape a ; write_testStr picture mix b a#b ; % ; t_name := '# chrStr_get_test example 2b r' ; t_input := -1 7 'e^2*(r·b)*r(rb)/|r2 - r1|^2*6*Bpe^2*Bep^2*(-3/2/π)' ; t_standard := '·' 1 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link 'should be OK, reading start at end of a unicode "·" i_end = 7' ; a := 2 pick t_input ; b:= tell gage shape a ; write_testStr picture mix b a#b ; % ; t_name := '# chrStr_get_test example 3 ∂/' ; t_input := 1 -2 '[·dl : E(r´,t´)] = -1/c*dp[dt : [dA : B(r,t) •n]] .and.[·dl : E(r´,t´)]' ; t_standard := (fault '?chrStr_get i_str out of range') 0 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link link 'This tests an out-of-bound i_str: ' (string -2) ', giving negative indices and error ' ; a := 2 pick t_input ; b:= tell gage shape a ; write_testStr picture mix b a#b ; % ; t_name := '# chrStr_get_test example 4' ; t_input := -1 67 'r/r/sinO *[ ∂/∂O(sinO*Ei(rp,t)·Pp) - ∂/∂P(Ei(rp,t)·O' ; t_standard := (fault '?chrStr_get i_str out of range') 0 ; t_result := chrStr_get t_input ; test_comment t_name t_input t_standard t_result ; write_testStr link link 'This tests an out-of-bound i_str: ' (string 67) ', > len: ' (string gage shape (third t_input)) ; a := 2 pick t_input ; b:= tell gage shape a ; write_testStr picture mix b a#b ; } # To check char positions : # a := 'e^2*(r·b)*r(rb)/|r2 - r1|^2*6*Bpe^2*Bep^2*(-3/2/π' # b:= tell gage shape a # write mix b a#b # All tests +-----+ 26Jun2018 12:29 changed chrList@i to (first chrList@i) in sym_stdForm conditionals +-----+ sym_stdForm IS OP str { LOCAL chrList i len symNew ; NONLOCAL chrs_blank ; % ; chrList := EACH string str ; len := gage shape chrList ; i := 0 ; % ; symNew := null ; WHILE (i < len) DO IF (in (first chrList@i) chrs_blank) THEN null ; ELSEIF (in (first chrList@i) '+-=<>') THEN symNew := link symNew ' ' chrList@i ' ' ; ELSEIF (in (first chrList@i) ':' ) THEN symNew := link symNew chrList@i ' ' ; ELSE symNew := link symNew chrList@i ; ENDIF ; i := i + 1 ; ENDWHILE ; % ; symNew } +-----+ >> now strings_alltest works *********************** 27Jun2018 str_to_unicodeList problems # IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := append chrList charrepList#i2s ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := append chrList charrepList#i3s ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := append chrList charrepList@i ; jump := 1 ; ENDIF ; ELSE chrList := append chrList charrepList@i ; jump := 1 ; ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := append chrList (solitary charrepList#i2s) ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := append chrList (solitary charrepList#i3s) ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := append chrList (solitary charrepList@i) ; jump := 1 ; ENDIF ; ELSE chrList := append chrList (solitary charrepList@i) ; jump := 1 ; ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := link chrList (solitary charrepList#i2s) ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := link chrList (solitary charrepList#i3s) ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := link chrList (solitary charrepList@i) ; jump := 1 ; ENDIF ; ELSE chrList := link chrList (solitary charrepList@i) ; jump := 1 ; ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := link (chrList (solitary charrepList#i2s)) ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := link (chrList (solitary charrepList#i3s)) ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := link (chrList (solitary charrepList@i)) ; jump := 1 ; ENDIF ; ELSE chrList := link (chrList (solitary charrepList@i)) ; jump := 1 ; ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := chrList (solitary charrepList#i2s) ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := chrList (solitary charrepList#i3s) ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := chrList (solitary charrepList@i) ; jump := 1 ; ENDIF ; ELSE chrList := chrList (solitary charrepList@i) ; jump := 1 ; ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := link chrList (solitary charrepList#i2s) ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := link chrList (solitary charrepList#i3s) ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := link chrList charrepList@i ; jump := 1 ; ENDIF ; ELSE chrList := link chrList charrepList@i ; jump := 1 ; -18 -124 -87 99 41 -18 -124 -87 ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := append chrList charrepList#i2s ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := append chrList charrepList#i3s ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := link chrList charrepList@i ; jump := 1 ; ENDIF ; ELSE chrList := link chrList charrepList@i ; jump := 1 ; -18 -124 -87 99 41 -18 -124 -87 ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := append chrList (solitary charrepList#i2s) ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := append chrList (solitary charrepList#i3s) ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := link chrList charrepList@i ; jump := 1 ; ENDIF ; ELSE chrList := link chrList charrepList@i ; jump := 1 ; -18 -124 -87 99 41 -18 -124 -87 ... IF (in charrepList#i2s unicodes_HFLN_2) THEN chrList := append chrList charrepList#i2s ; jump := 2 ; chr_found := l ; ENDIF ; ENDIF ; IF (AND (NOT chr_found) (n_chars >= 3)) THEN i3s := 3 take i_steps ; IF (in charrepList#i3s unicodes_HFLN_3) THEN chrList := append chrList charrepList#i3s ; jump := 3 ; chr_found := l ; ENDIF ; ENDIF ; %I don't use any [1,4]-byte unicodes at present ; IF (NOT chr_found) THEN chrList := link chrList charrepList@i ; jump := 1 ; ENDIF ; ELSE chrList := link chrList charrepList@i ; jump := 1 ; -18 -124 -87 99 41 -18 -124 -87 ... # EACH char chrList +-----+ 27Jun2018 15:19 test_comment - Problem with equalities : -->[nextv] t_standard +---+-+-+---+ ||c|)|| +---+-+-+---+ -->[nextv] t_result +---+-+-+---+ ||c|)|| +---+-+-+---+ -->[nextv] EACH shape t_standard t_result +-+-+ |4|4| +-+-+ -->[nextv] = t_standard@1 t_result@1 o -->[nextv] EACH isstring t_standard@1 t_result@1 lo -->[nextv] EACH ischar t_standard@1 t_result@1 ol -->[nextv] EACH isstring t_standard@0 t_result@0 lo -->[nextv] EACH ischar t_standard@0 t_result@0 ol qnial> str_to_unicodeList_test #+-----+ str_to_unicodeList_test, Wed Jun 27 15:26:30 2018 # str_to_unicodeList_test example 1 : OK - result matches standard t_input, t_standard, t_result = c) +---+-+-+---+ ||c|)|| +---+-+-+---+ +---+-+-+---+ ||c|)|| +---+-+-+---+ # str_to_unicodeList_test example 2 : OK - result matches standard t_input, t_standard, t_result = Bi(r,v,t) = q/c*(vrp)/rs^3 = (v/c)E0(r,t) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+ |B|i|(|r|,|v|,|t|)| |=| | |q|/|c|*|(|v||r|p|)|/|r|s|^|3| | |=| | |(|v|/|c|)||E|0|(|r|,|t|)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+ |B|i|(|r|,|v|,|t|)| |=| | |q|/|c|*|(|v||r|p|)|/|r|s|^|3| | |=| | |(|v|/|c|)||E|0|(|r|,|t|)| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+ # str_to_unicodeList_test example 3 : OK - result matches standard t_input, t_standard, t_result = e^2*(r·b)*r(rb)/|r2 - r1|^2*6*Bpe^2*Bep^2*(-3/2/π) +-+-+-+-+-+-+--+-+-+-+-+---+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+ |e|^|2|*|(|r|·|b|)|*|r||(|r||b|)|/|||r|2| |-| |r|1|||^|2|*|6|*|B|p|e|^|2|*|B|e|p|^|2|*|(|-|3|/|2|/|π|)| +-+-+-+-+-+-+--+-+-+-+-+---+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+ +-+-+-+-+-+-+--+-+-+-+-+---+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+ |e|^|2|*|(|r|·|b|)|*|r||(|r||b|)|/|||r|2| |-| |r|1|||^|2|*|6|*|B|p|e|^|2|*|B|e|p|^|2|*|(|-|3|/|2|/|π|)| +-+-+-+-+-+-+--+-+-+-+-+---+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+ # str_to_unicodeList_test example 4 : OK - result matches standard t_input, t_standard, t_result = [·dl : E(r´,t´)] = -1/c*dp[dt : [dA : B(r,t) •n]] .and.[·dl : E(r´,t´)] +---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+ ||[|·|d|l| |:| |E|(|r|´|,|t|´|)|]| | |=| |-|1|/|c|*|d|p|[|d|t| |:| ||[|d|A| |:| |B|(|r|,|t|)| | |•|n|]|]| |.|a|n|d|.||[|·|d|l| |:| |E|(|r|´|,|t|´|)|]| +---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+ +---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+ ||[|·|d|l| |:| |E|(|r|´|,|t|´|)|]| | |=| |-|1|/|c|*|d|p|[|d|t| |:| ||[|d|A| |:| |B|(|r|,|t|)| | |•|n|]|]| |.|a|n|d|.||[|·|d|l| |:| |E|(|r|´|,|t|´|)|]| +---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---+-+-+-+-+-+-+-+-+-+---+-+--+-+-+-+-+-+-+-+-+--+-+-+--+-+-+ # str_to_unicodeList_test example 5 : OK - result matches standard t_input, t_standard, t_result = r/r/sinO *[ ∂/∂O(sinO*Ei(rp,t)·Pp) - ∂/∂P(Ei(rp,t)·O +-+---+-+-+-+-+-+-+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+--+-+ |r||/|r|/|s|i|n|O| |*|[| | |∂|/|∂|O|(|s|i|n|O|*|E|i|(|r|p|,|t|)|·|P|p|)| |-| |∂|/|∂|P|(|E|i|(|r|p|,|t|)|·|O| +-+---+-+-+-+-+-+-+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+--+-+ +-+---+-+-+-+-+-+-+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+--+-+ |r||/|r|/|s|i|n|O| |*|[| | |∂|/|∂|O|(|s|i|n|O|*|E|i|(|r|p|,|t|)|·|P|p|)| |-| |∂|/|∂|P|(|E|i|(|r|p|,|t|)|·|O| +-+---+-+-+-+-+-+-+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+--+-+-+-+-+-+-+---+-+---+-+-+-+-+-+-+-+-+-+-+--+-+ # str_to_unicodeList_test example 6 : OK - result matches standard t_input, t_standard, t_result = rDEFG~!@#∫∮∬∂ΝΞΟΠΡεζηθικ +-+---+-+-+-+-+-+-+-+-+---+---+---+---+--+--+--+--+--+--+--+--+--+--+--+ |r||D|E|F|G|~|!|@|#|∫|∮|∬|∂|Ν|Ξ|Ο|Π|Ρ|ε|ζ|η|θ|ι|κ| +-+---+-+-+-+-+-+-+-+-+---+---+---+---+--+--+--+--+--+--+--+--+--+--+--+ +-+---+-+-+-+-+-+-+-+-+---+---+---+---+--+--+--+--+--+--+--+--+--+--+--+ |r||D|E|F|G|~|!|@|#|∫|∮|∬|∂|Ν|Ξ|Ο|Π|Ρ|ε|ζ|η|θ|ι|κ| +-+---+-+-+-+-+-+-+-+-+---+---+---+---+--+--+--+--+--+--+--+--+--+--+--+ >> OK, now it seems to work ********************************* 28Aug2018 17:04 problem with str_to_unicodeList -->[nextv] str ∮(·dl, over ∂Σ : E) = ∫(·dA, over : dp[dt : B] ) + Ψ -->[nextv] charrepList 226 136 174 40 194 183 100 108 44 32 111 118 101 114 32 226 136 130 206 163 32 58 32 69 41 32 61 32 226 136 17 1 40 194 183 100 65 44 32 111 118 101 114 32 58 32 100 112 91 100 116 32 58 32 66 93 32 41 32 43 32 206 168 >> YIKES!! unicodes are no longer negative??? 28Aug2018 17:34 The Eaarth just moved! Day became night! A day of 36 hours! charrep now returns integers > 127 for unicodes, not negatives!!! WHY??? ******************************* 30Aug2018 13:06 str_to_unicodeList IS OP str check if LMDE2 (in addition to RaspPi) now uses UNSIGNED bytes for chars ; change : IF (OR (charrepList@i < 0) (charrepList@i > 127)) THEN to : IF (charrepList@i > 127) THEN . Now run "symExtract_alltest" in "symbols extract- tests.ndf" +-----+ Summary of test results : /media/bill/PROJECTS/Qnial/code develop_test/alltest symExtract 180830 13h11m46s.txt, date= 180830 13h12m # symParen_contents_test example 1 : OK - result matches standard # symParen_contents_test example 2 : OK - result matches standard # symParen_contents_test example 3 : OK - result matches standard # symParen_contents_test example 4 : OK - result matches standard # symParen_contents_test example 5 : OK - result matches standard # syms_inParens_test example 1 : OK - result matches standard # syms_inParens_test example 2 : OK - result matches standard # syms_inParens_test example 3 : OK - result matches standard # syms_inParens_test example 4 : OK - result matches standard # syms_inParens_test example 5 : OK - result matches standard # symExtract_stringPosn_test example 4 : OK - result matches standard # symExtract_string_test example 1 : OK - result matches standard # symExtract_string_test example 4 : OK - result matches standard # symExtract_string_test example 9 : OK - result matches standard # symExtract_stringPosn_test example 1 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 2 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 3 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 5 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 6 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 7 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 8 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 9 : FAILED - result does NOT match standard # symExtract_stringPosn_test example 10 : FAILED - result does NOT match standard # symExtract_string_test example 2 : FAILED - result does NOT match standard # symExtract_string_test example 3 : FAILED - result does NOT match standard # symExtract_string_test example 5 : FAILED - result does NOT match standard # symExtract_string_test example 6 : FAILED - result does NOT match standard # symExtract_string_test example 7 : FAILED - result does NOT match standard # symExtract_string_test example 8 : FAILED - result does NOT match standard # symExtract_string_test example 10 : FAILED - result does NOT match standard # symExtract_string_test example 21 : FAILED - result does NOT match standard # symExtract_string_test example 22 : FAILED - result does NOT match standard # symExtract_string_test example 23 : FAILED - result does NOT match standard +-----+ >> oops! loks like LMDE2 is SIGNED unicode? So, in str_to_unicodeList change : IF (charrepList@i > 127) THEN to : IF (charrepList@i < 0) THEN ... >> now ALL tests work!! 30Aug2018 13:19 LMDE2 uses SIGNED bytes, RaspPi UNSIGNED in "str_to_unicodeList IS OP str " definition : LMDE2 : IF (charrepList@i < 0) THEN RaspPi : IF (charrepList@i > 127) THEN >> I need to code this automatically someday For now, use : IF (OR (charrepList@i < 0) (charrepList@i > 127)) THEN # enddoc