#]
#] *********************
#] "$d_Qtest"'fileops/file_ops- develop.txt'
www.BillHowell.ca 13Apr2018 initial
# 01Oct2021 -???? items pulled from :
# link d_Qtest 'fileops/Qndfs optr symbol changes - develop.txt' - change optr names
ToDos
27Oct2020 move to projects-mini for later ..
_Galactic rays and evolution - life, the mind, civilisation, economics, financial markets.html
Mythology
ETC, ETC
24************************24
24************************24
# Table of Contents, generated in :
$ grep "^#]" "$d_Qtest""fileops/file_ops- develop.txt" | sed 's/^#\]/ /' >"$d_Qtest"'fileops/file_ops- develop TblOfContents.txt'
24************************24
08********08
#] ??Apr2023
08********08
#] ??Apr2023
08********08
#] ??Apr2023
08********08
#] ??Apr2023
08********08
#] 10Apr2023
+-----+
strL_writeTo_pth
$ find "$d_Qndfs" -maxdepth 4 -type f -name "*" | grep --invert-match "z_Old\|z_Archive" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "strL_writeTo_pth" "FILE" | grep "IS OP"
/home/bill/web/Qnial/MY_NDFS/file_ops.ndf:49:#l strL_writeTo_pth IS OP strL p_out { fault '?strL_writeTo_pth undefined (QNial setup header.ndf)' }
>> nuts, I didn't write this yet!!??
adapt : strL_write_fout IS OP strL fout
to : strL_writeTo_pth IS OP strL p_out
+-----+
winTitle_get_winID winTitle
$ find "$d_Qndfs" -maxdepth 4 -type f -name "*" | grep --invert-match "z_Old\|z_Archive" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "winTitle_get_winID" "FILE" | grep "IS OP"
+--+
/home/bill/web/Qnial/MY_NDFS/windows.ndf:29: winTitle_get_winID IS OP winIDorTitle - returns a list of winIDs having the name requested
/home/bill/web/Qnial/MY_NDFS/windows.ndf:54:#l winTitle_get_winID IS OP winIDorTitle { fault '?winTitle_get_winID undefined (QNial setup header.ndf)' }
/home/bill/web/Qnial/MY_NDFS/windows.ndf:97:#] winTitle_get_winID IS OP winIDorTitle - returns a list of winIDs having the name requested
/home/bill/web/Qnial/MY_NDFS/windows.ndf:101: winTitle_get_winID IS OP winIDorTitle
+--+
+-----+
olde code
IF flag_debug THEN write 'loading checkExists_pthOrStdout' ; ENDIF ;
# checkExists_pthOrStdout IS OP pthOrStdout - returns EITHER [result, fault]
# result - ("outputStr outputStr) for redirecting output, including stdout option
# fault - if pth doesn't exist
# 06Nov2021 revamped
checkExists_pthOrStdout IS OP pthOrStdout
{ LOCAL resFltL ;
IF (= 1 pthOrStdout) THEN
resFltL := "outputStr ' ' ;
ELSE resFltL := pth_typ_checkExists pthOrStdout "pinn ;
IF (NOT isfault resFltL) THEN
resFltL := "outputStr (link ' >"' pthOrStdout '" ') ;
ENDIF ;
ENDIF ;
resFltL
}
#l pth_typ_checkExists IS OP pth typ { fault '?pth_typ_checkExists undefined (QNial setup header.ndf)' }
08********08
#] 30Mar2023 sub_inTop_ary - not working (types.ndf)
08********08
#] 30Mar2023 pth_typ_exists_test after fixing ary_adrCompatible_testSimple
qnial> pth_typ_exists_test
pth_typ_exists 10 25
Changed pthTypPhr must be updated :
file_ops.ndf :
pth_typ_exists IS OP pth typ
IF (NOR (= "pinn typ) (= "padd typ) (= "pnew typ) (= "povr typ)
(= "dinn typ) (= "dnew typ)
"$d_Qtest"'fileops/fileops- test.ndf'
d[old, new] p[old new, ovr]
I fixed up and augmented tests
qnial> pth_typ_exists_test
pth_typ_exists 30 36
>> awesome, most of the tests work
+--+------------+---------------------------------------+----+------+------+----------------------------------
|OK|test# |[dir, pth] |type|stndrd|calc |comments
+--+------------+---------------------------------------+----+------+------+----------------------------------
| o|01_d_noExist|+-------------------------+ |dinn|?pte1 | l|should fail - dir doesn"t exist
| | ||test input files/noExist/| | | | |
| | |+-------------------------+ | | | |
+--+------------+---------------------------------------+----+------+------+----------------------------------
| o|02_d_noExist|+-------------------------+ |dnew|?pte11|?pte10|should fail - dirUp doesn"t exist
| | ||test input files/noExist/| | | | |
| | |+-------------------------+ | | | |
+--+------------+---------------------------------------+----+------+------+----------------------------------
| o|05_d_noExist|+-------------------------+ |povr|?pte9 |?pte3 |should fail - dirUp doesn"t exist
| | ||test input files/noExist/| | | | |
| | |+-------------------------+ | | | |
+--+------------+---------------------------------------+----+------+------+----------------------------------
| o|06_d_noExist|+-------------------------+ |padd|?pte9 |?pte3 |should fail - dirUp doesn"t exist
| | ||test input files/noExist/| | | | |
| | |+-------------------------+ | | | |
+--+------------+---------------------------------------+----+------+------+----------------------------------
| o|03_p_noExist|+---------------------------+ |pinn|?pte5 | l|should fail - pth doesn"t exist
| | ||test input files/strngs.ndf| | | | |
| | |+---------------------------+ | | | |
+--+------------+---------------------------------------+----+------+------+----------------------------------
| o|04_p_noExist|+---------------------------+ |pnew| l|?pte7 |should be OK - upDir exists, pth n
| | ||test input files/strngs.ndf| | | | |
| | |+---------------------------+ | | | |
+--+------------+---------------------------------------+----+------+------+----------------------------------
08********08
#] 26Mar2023 pth_typ_exists not working again
qnial> d_mktDat := '/home/bill/PROJECTS/economics, markets/'
/home/bill/PROJECTS/economics, markets/
qnial> pYrFrac := link d_mktDat 'mktDat M/yrFrac/yrFrac CADUSD TV.txt'
/home/bill/PROJECTS/economics, markets/mktDat M/yrFrac/yrFrac CADUSD TV.txt
qnial> pth_typ_exists pYrFrac "pinn
l
>> WRONG!?!?!?
NUTS - problem with QNial [sym, optr, etc] changes ongoing...
at least this is safer than to over-write good files!
Run my tests : pth_typ_exists_test in "$d_Qtest"'fileops/fileops- test.ndf'
qnial> pth_typ_exists_test
pth_typ_exists 10 25
>> yeesh, still the old terminology [pnew, pold, etc] - not really, but very incomplete!
try :
qnial> pth_typ_exists pYrFrac "pinn
l
>> NUTS
flag_break it
lq_fileops
fonn
>> STUPID! I had a chain of IF, ELSEIFs, but they must be independent!?
"$d_temp"'hostCmdRslt.txt' :
find: ‘/home/bill/PROJECTS/economics, markets/mktDat M/yrFrac/yrFrac CADUSD TV.txt’: No such file or directory
>> but, filFind = o ???
sub_inTop_ary - wrong, see : "$d_Qtest"'arrays/arrays- develop.txt'
08********08
#] 15Nov2021
+-----+
olde code from link d_Qndfs 'QNial symbol changes.ndf' - no sense having a duplicate
# loaddefs link d_Qndfs 'QNial symbol changes.ndf'
# QNial_optr_symbol_changes_loaddef_postStartup
IF flag_debug THEN write 'loading pthL_searchReplaceL_sedOverwrite_povrOrStdoutNull' ; ENDIF ;
#] pthL_searchReplaceL_sedOverwrite_povrOrStdoutNull IS OP
# pthL searchReplaceL povrOrStdoutNull -
# [multiple, "parallel"]cart[searchReplaceL, pthL] sed search&replace, (pout = 1) for stdOut
# original pthL is backed up, then over-written, returns null if no errors, faultL otherwise
# assumes "problematic" sed chrs have already been escaped
# if problematic sedChrs in searchReplaceL, customized [sed, QNial [slow, limited]!] approaches required
# this is all going to go VERY badly if strPairL isn't a list!!
# for just one pair, make it into solitary, eg : ['string1' 'string2']
# 15Oct2021 initial, 20-24Oct2021 major revamp
# IF flag_break THEN BREAK ; ENDIF ;
pthL_searchReplaceL_sedOverwrite_povrOrStdoutNull IS OP pthL searchReplaceL povrOrStdoutNull
{ LOCAL cmd faultL p_temp searchReplaceL searchL replaceL ;
NONLOCAL d_temp ;
% ;
faultL := null ;
% faultL := check_argL link ['pinn_searchL_options_grepTo_povrOrStdout']
(cols mix EACH [pass, execute] 'pinn' 'searchL' 'options' 'povrOrStdoutNull') ;
% ;
IF flag_break THEN BREAK ; ENDIF ;
p_temp := link d_temp 'pthL_searchReplaceL_options_sedOverwrite tempt.txt' ;
searchReplaceL := lists_sortupCullOn1st searchReplaceL ;
searchL replaceL := rows mix searchReplaceL ;
% ;
faultL := null ;
% faultL := link faultL pth_typ_checkExists pthL "p_old ;
% faultL := link faultL (EACH str_checkIs (link searchL replaceL)) ;
% 23Oct2021 dummy optr - returns null ;
% faultL := link result (EACH searchL_checkNoEscape searchL ) ;
% faultL := link result (EACH replaceL_checkNoEscape replaceL) ;
% faultL := link faultL pth_typ_exists p_temp "p_new ;
% ;
IF (= null faultL) THEN
% no backups - must do prior to calling this optr, not normally desired for searches ;
searchL replaceL := rows mix searchReplaceL ;
sedExpr := sub_insertBetweenThenLink_aryL '\|' searchL ;
write sedExpr ;
cmd := link 'grep ' options ' "' sedExpr '" "' pinn '" ' povrOrStdoutNull ;
host cmd ;
ENDIF ;
faultL
}
# olde code
result := link result (EACH strSedSearchL_checkNoEscape strSedSearchL ) ;
result := link result (EACH strSedReplaceL_checkNoEscape strSedReplaceL) ;
# 10Sep2021 perhaps I should reincarnate an [alternate, more general, vastly slower] QNial-only version?
# especially for sed restricted : search $.*/[\]^ replace &\/\n
# sed [search, replace] to change pthL
Replace userSymNames in ndf files :
IF flag_debug THEN write '+-----+' ; ENDIF ;
IF flag_debug THEN write 'searchReplace userSymNames in ndf files : [optr, expr]cart[Decl, Desc]L ' ; ENDIF ;
# loaddefs link d_Qndfs 'QNial symbol changes.ndf'
# QNial_optr_symbol_changes_loaddef_postStartup
# IF flag_break THEN BREAK ; ENDIF ;
IF flag_debug THEN write 'loading pinn_pStrPairL_replaceToPtmp' ; ENDIF ;
#] pinn_pStrPairL_replaceToPtmp IS OP pinn pout - temporary optr, fixes while working on "QNial core optrs"
pinn_pStrPairL_replaceToPtmp IS OP pinn pStrPairL
{ LOCAL cmd ptmp wordOldNewL wordOption
sed_expr1 sed_expr2 sed_expr3 sed_expr4 ;
NONLOCAL d_temp ;
% ;
wordOldNewL := pinn_readStrPairL pStrPairL ;
fname := pthNm_extract_fname pinn ;
ptmp := link d_temp fname ;
% ;
IF flag_break THEN BREAK ; ENDIF ;
wordOption := '\b' ;
sed_expr1 := (link 's|' wordOption) EACHRIGHT link (EACH first wordOldNewL) ;
sed_expr2 := sed_expr1 EACHLEFT link (link wordOption '|') ;
sed_expr3 := sed_expr2 EACHBOTH link ((EACH second wordOldNewL) EACHLEFT link '|;') ;
sed_expr4 := front link sed_expr3 ;
% ;
cmd := link 'sed "s/' sed_expr4 '" "' pinn '" >"' ptmp '" ' ;
write cmd ;
% host cmd ;
}
# pinn_pStrPairL_replaceToPtmp (link d_Qndfs 'strings symbolOldNew strings.txt') (link
#] str_replaceIn_pth testing
# 20Nov2020 test created
IF flag_debug THEN write '+-----+' ; ENDIF ;
IF flag_debug THEN write 'str_replaceIn_pth testing ' ; ENDIF ;
# loaddefs link d_Qtest 'fileops/fileops- test.ndf'
IF flag_debug THEN write 'loading str_replaceIn_pth_output' ; ENDIF ;
#] str_replaceIn_pth_output IS OP i_test pinn - common optr to execute & ouput diff to log file
# 20Nov2020 initial, adapted from 'Website updates- tests.ndf'
str_replaceIn_pth_output IS OP i_test strOld strNew pstd pinn
{ LOCAL flog p_log p_temp ;
NONLOCAL d_Qtest ;
p_log := link d_Qtest 'file_ops- test log.txt' ;
flog := open p_log "a ;
flog EACHRIGHT writefile
'........'
(link '# str_replaceIn_pth_test example #' (string i_test))
pinn
;
close flog ;
% ;
p_temp := link d_temp 'str_replaceIn_pth temp.txt' ;
% Note that for tests, flag_backup is false ;
str_replaceIn_pth o '' strOld strNew pinn ;
host (link 'diff --width=85 "' pstd '" "' p_temp '" --suppress-common-lines | grep ' chr_apo '^>' chr_apo ' | sed ' chr_apo 's/^>\ //' chr_apo ' >>"' p_log '"') ;
host link 'echo "" >>"' p_log '"' ;
}
# loaddefs link d_Qtest 'fileops/fileops- test.ndf'
IF flag_debug THEN write 'loading str_replaceIn_pth_test' ; ENDIF ;
#] str_replaceIn_pth_test IS OP - test pinn link fixes, write diff to log file
# 20Nov2020 initial, adapted from 'Website updates- tests.ndf'
str_replaceIn_pth_test IS
{ LOCAL flog p_log i_test strOld strNew pstd pinn ;
NONLOCAL d_Qtest ;
p_log := link d_Qtest 'file_ops- test log.txt' ;
flog := open p_log "a ;
flog EACHRIGHT writefile '' ''
'**********************************'
(link 'str_replaceIn_pth_test, ' timestamp)
'' ;
close flog ;
i_test := 0 ;
% ;
i_test := i_test + 1 ;
strOld strNew := '../../../' '[#=; backtrack ;=#]' ;
pstd := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html str_replaceIn_pth.html' ;
pinn := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html' ;
str_replaceIn_pth_output i_test strOld strNew pstd pinn ;
}
# uncomment to run all tests (insert line below, remove extra line below next test definition)
str_replaceIn_pth_test IS
{ LOCAL flog p_log i_test strOld strNew pstd pinn ;
NONLOCAL d_Qtest ;
p_log := link d_Qtest 'file_ops- test log.txt' ;
flog := open p_log "a ;
flog EACHRIGHT writefile '' ''
'**********************************'
(link 'str_replaceIn_pth_test, ' timestamp)
'' ;
close flog ;
i_test := 0 ;
% ;
i_test := i_test + 1 ;
strOld strNew := '!!linkError!!' '[#=; backtrack ;=#]' ;
pstd := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html str_replaceIn_pth.html' ;
pinn := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html' ;
str_replaceIn_pth_output i_test strOld strNew pstd pinn ;
% ;
i_test := i_test + 1 ;
strOld strNew := '!!linkError!!' '[#=; backtrack ;=#]' ;
pstd := link d_Qtest 'test- page Howell - blog.html str_replaceIn_pth.html' ;
pinn := link d_Qtest 'test- page Howell - blog.html' ;
str_replaceIn_pth_output i_test strOld strNew pstd pinn ;
% ;
i_test := i_test + 1 ;
strOld strNew := '!!linkError!!' '[#=; backtrack ;=#]' ;
pstd := link d_Qtest 'test- Howell - corona virus.html str_replaceIn_pth.html' ;
pinn := link d_Qtest 'test- Howell - corona virus.html' ;
str_replaceIn_pth_output i_test strOld strNew pstd pinn ;
% ;
i_test := i_test + 1 ;
strOld strNew := '!!linkError!!' '[#=; backtrack ;=#]' ;
pstd := link d_Qtest 'test- _Lies, damned lies, and scientists.html str_replaceIn_pth.html' ;
pinn := link d_Qtest 'test- _Lies, damned lies, and scientists.html' ;
str_replaceIn_pth_output i_test strOld strNew pstd pinn ;
% ;
i_test := i_test + 1 ;
strOld strNew := '!!linkError!!' '[#=; backtrack ;=#]' ;
pstd := link d_Qtest 'test- HELP.html str_replaceIn_pth.html' ;
pinn := link d_Qtest 'test- HELP.html' ;
str_replaceIn_pth_output i_test strOld strNew pstd pinn ;
}
IF flag_debug THEN write 'loading pinnL_options_searchL_grepCountLL_povrOrScreen' ; ENDIF ;
#] pinnL_options_searchL_grepCountLL_povrOrScreen IS OP pinnL options povrOrStdout flag_word - count each of [word, str]L
#] hits in each path listed in a file
#] uses grep therefore may not simply handle grep restricted : search $.*/[\]^
# 24Oct2020 initial, I am always searching...
# 11Sep2021 pinnL_options_searchL_grepCountLL_povrOrScreen I NOT yet fixed this, or renamed its usage in other optrs
# 09Oct2021 just use grep where problematicChrs not present?
# 06Nov2021 revamped
# see strings.ndf for : 'str_chrRegExprL_checkNoEscape IS OP str chrRegExprL'
# IF flag_break THEN BREAK ; ENDIF ;
pinnL_options_searchL_grepCountLL_povrOrScreen IS OP pinnL options searchL povrOrStdout
{ LOCAL grepOptions resFltL ;
NONLOCAL chr_apo ;
% ;
resFltL := null ;
FOR pinn WITH pinnL DO
resFltL := link resFltL (pth_typ_exists pinn "pold) ;
ENDFOR ;
resFltL := link resFltL (pth_typ_exists povrOrStdout "pold) ;
% ;
IF (= null resFltL) THEN
% build common partial regexp for grep, flag_word forces word-only matches ;
options := link ' -c' options ;
pinn_options_grepOptions_grepTo_povrOrStdout pinn options grepOptions povrOrStdout ;
ENDIF ;
% ;
resFltL
}
#! 09Oct2021: chrL_anyIn_str_z IS OP z_chrA_anyIn_strA
#! 11Sep2021 for deletion : pListFiles_findCountsBY_searchL_z IS z_pinnL_options_findCountsEach_povr
08********08
#] 21Oct2021 these old optrs are really NOT the way to go! They were removed from 'file_s.ndf' :
#] +-----+
#] [NOT, SELDOM] used backup [pth, dir]s :
IF flag_debug THEN write 'loading pth_backupDatedTo_dir' ; ENDIF ;
#] pth_backupDatedTo_dir IS OP pth dirBackup - backup dated version of a file to dirBackup
# 24Oct2020 new variant of [pth_backupDated, pth_backupDated_delete]
# For [single, small number of] path backups
# PROBLEM - Often, different files in different directories have the same name. At present, th IS OP erator overwrites backups of [earlier, same-name but different] files.
# some fileNames don't have extensions!
# In the future - special naming (eg part of original dirName) to avoid this, with checks.
IF flag_break THEN BREAK ; ENDIF ;
pth_backupDatedTo_dir IS OP pth dirBackup
{ LOCAL fname p_out ;
IF (NOT pth_typ_exists ("d_old dirBackup)) THEN
host link 'mkdir ' dirBackup ;
ENDIF ;
IF (NOT AND (EACH pth_typ_exists ("p_old pth) ("d_old dirBackup)))
THEN EACH write
'?pth_backupDatedTo_dir file unknown error, OR [pth dirBackup] : '
pth
dirBackup
'' ;
ELSE
fname := pth_extract_fname pth ;
p_out := link dirBackup timestamp_YYMMDD_HMS ' ' fname ;
host link 'cp -up "' pth '" "' p_out '" ' ;
ENDIF ;
}
# pth_backupDatedTo_dir (link d_Qndfs 'strings.ndf') (link d_Qndfs 'z_Archive/201024 backups/')
# pth_backupDatedTo_dir (link d_Qndfs 'strings.ndf') (link d_Qndfs 'z_Archive/201027 backups/')
# host_result link 'ls -1 "/media/bill/SWAPPER/Website - raw/z_Archive/201024 backups" '
# dirBackup_restoreTo_pths o (link d_webRaw 'z_Archive/201117 17h00m21s backups/') htmlpthsSortedBypth
IF flag_debug THEN write 'loading pth_backupTo_dir' ; ENDIF ;
#] pth_backupTo_dir IS OP pth dirBackup - NOT USED normally, clumsy
# 20Nov2020 initial based on
# For massive changes in [deep, broad] directories, it is impractical to backup each file in it's own directory. Afterwards, manual cleanout of directories would not happen. So the approach here is to save the file, with its original name, in a [flat, dated] directory whose name is provided.
# PROBLEM - Often, different files in different directories have the same name. At present, th IS OP erator overwrites backups of [earlier, same-name but different] files.
# some fileNames don't have extensions!
# In the future - special naming (eg part of original dirName) to avoid this, with checks.
IF flag_break THEN BREAK ; ENDIF ;
pth_backupTo_dir IS OP pth dirBackup
{ LOCAL fname p_out ;
IF (NOT AND (EACH pth_typ_exists ("p_old pth) ("d_old dirBackup)))
THEN EACH write
'?pth_backupDatedTo_dir file unknown error, OR [pth dirBackup] : '
pth
dirBackup
'' ;
ELSE
fname := pth_extract_fname pth ;
p_out := link dirBackup fname ;
host link 'cp -up "' pth '" "' p_out '" ' ;
ENDIF ;
}
IF flag_debug THEN write 'loading pth_backupDatedToSameDir' ; ENDIF ;
#] pth_backupDatedToSameDir IS OP pth - DON'T USE normally, clutters up working dir
IF flag_break THEN BREAK ; ENDIF ;
pth_backupDatedToSameDir IS OP pth
{ LOCAL dir base extn p_new ;
dir base extn := pth_extract_dirBaseExtn pth ;
p_new := link dir timestamp_YYMMDD_HMS ' ' base '.' extn ;
host link 'cp "' pth '" "' p_new '" ' ;
}
#! 06Sep2021 for deletion : file_backupDated_retain_z IS z_pth_backupDatedToSameDir
#! 06Sep2021 for deletion : pth_backupDated_retain_z IS z_pth_backupDatedToSameDir
# f_test := '/media/bill/SWAPPER/Website - raw/4_website htmlFileList.txt'
# file_dateTimeStamp f_test
# pth_backupDated_retain f_test
%EACH write pth p_new ;
IF flag_debug THEN write 'loading pth_backupDated_delete' ; ENDIF ;
#] pth_backupDated_delete IS OP pth - DON'T USE normally!! rename a file with date precursor
pth_backupDated_delete IS OP pth
{ LOCAL dir base extn p_new ;
dir base extn := pth_extract_dirBaseExtn pth ;
fname := pth_extract_fname pth ;
p_new := link dir timestamp_YYMMDD ' ' fname ;
host link 'mv "' pth '" "' p_new '" ' ;
}
#! 06Sep2021 for deletion : file_backupDated_delete_z_z IS z_z_pth_backupDated_delete
# f_test := '/media/bill/SWAPPER/Website - raw/4_website htmlFileList.txt'
# file_dateTimeStamp f_test
# pth_backupDated_delete f_test
%EACH write pth p_new ;
#] +-----+
#] crappy old tools for using [find, grep, sed] :
IF flag_debug THEN write 'loading pinn_sedPretreat_poutFixed' ; ENDIF ;
#] pinn_sedPretreat_poutFixed IS OP pinn - pre-process pinn to handle [apo, quote,'&\/\n'])
# uses unchanging poutFixed = d_temp 'sed_file_pre translated.txt'
# via bash script, used by "symTranslate_file" in "$d_Qndfs""symbols/symbols translate.ndf"
# 05May2018 initial
# 23Oct2020 I removed [grep, sed] from strings.ndf - pasted to link d_Qtests 'strings- develop.txt'
# [pinn_sedPretreat_poutFixed, sed_file_postreat] will have to be redefined
# 11Sep2021 BROKEN!!! - will have to fix when I next work on "$d_Qndfs""symbols/symbols translate.ndf"
pinn_sedPretreat_poutFixed IS OP pinn
{ LOCAL pname_cpy ;
pname_cpy := link d_temp 'sed_file_pre input.txt' ;
host link 'cp -up "' pinn '" "' pname_cpy '" ' ;
host link 'bash "' d_temp 'strings pinn_sedPretreat_poutment.sh' ;
}
#! 11Sep2021 for deletion : sed_file_pretreat_z IS z_pinn_sedPretreat_poutFixed
IF flag_debug THEN write 'loading pinnFixed_sedPostTreat_pout' ; ENDIF ;
#] pinnFixed_sedPostTreat_pout IS OP pout - translate pname_inn for QNial use
# mostly [apostrophes, quotes,'&\/\n'], via bash script
# uses unchanging pinnFixed = d_temp 'sed_file_pre translated.txt'
# uses unchanging poutFixed = d_temp 'sed_file_pre translated.txt'
# used by "symTranslate_file" in "/media/bill/SWAPPER/Qnial/MY_NDFS/symbols translate.ndf"
# 05May2018 initial
# 11Sep2021 doesn't look like this will work now!!
# will have to fix when I next work on "$d_Qndfs""symbols/symbols translate.ndf"
pinnFixed_sedPostTreat_pout IS OP pout
{ LOCAL pname_pst ;
pname_pst := link d_temp 'sed_file_pre translated.txt' ;
host link 'bash "' d_temp 'strings sed_file_postreatment.sh' ;
host link 'cp "' pname_pst '" "' pout '" ' ;
}
#! 09Oct2021 : sed_file_postreat_z IS pinnFixed_sedPostTreat_pout
#] +-----+
#] Specialized stuff for 2017 IJCNN Alaska :
IF flag_debug THEN write '+-----+' ; ENDIF ;
IF flag_debug THEN write 'Specialized stuff for 2017 IJCNN Alaska' ; ENDIF ;
IF flag_debug THEN write 'loading line_charcodes' ; ENDIF ;
#] line_charcodes IS OP pth line_start line_end - convert ASCII file lines to charcodes over interval
line_charcodes IS OP pth line_start line_end
{ fin := open pth "r ;
line_count := 1 ;
flag_stop := o ;
flag_in := o ;
WHILE (NOT OR flag_stop (isfault (text_line := readfile fin))) DO
IF line_count = line_end THEN flag_stop := l ;
ELSEIF line_count = line_start THEN flag_in := l ;
ENDIF ;
IF flag_in THEN
write text_line ;
write EACH charrep text_line ;
ENDIF ;
line_count := line_count + 1 ;
ENDWHILE ;
}
IF flag_debug THEN write 'loading rename_files' ; ENDIF ;
#] rename_files IS [later ... OP dname fname_oldPattern fname_newPattern] - rename a bunch of files
# (would like reg_exprn, but too long to program)
# At this stage is just simple one-way use of formulae
# set up for IJCNN2017
rename_files IS
{ dname := '/home/bill/2017 IJCNN Alaska/' ;
FOR i WITH (6 + (tell (938 - 5))) DO
num := string i ;
lead_zeros := (4 - (gage shape num)) reshape '0' ;
fname_start := link 'N-' lead_zeros num '.pdf' ;
fname_end := link 'N' lead_zeros num '.pdf' ;
host (link 'mv "' dname fname_start '" "' dname fname_end '"' ) ;
host 'sleep 0.1s' ;
ENDFOR ;
}
IF flag_debug THEN write 'loading pthL_moveEach' ; ENDIF ;
#] pthL_moveEach IS - move selected files listed in text file
# example - set up for IJCNN2017 - CrossCheck maximum 200 files per zip file
pthL_moveEach IS
{ LOCAL d_base d_IJCNN2017 d_moved fin fname i j num_str plist text_line ;
d_base := '/home/bill/2017 IJCNN Alaska/' ;
d_IJCNN2017 := '/media/bill/SWAPPER/2017 IJCNN Anchorage/Publications/plagiarism - CrossCheck/' ;
% plist := link d_IJCNN2017 '170126 Accepted now.txt' ;
plist := link d_IJCNN2017 '170130 Accepted since 170126.txt' ;
fin := open plist "r ;
IF flag_debug THEN write 'fin = ' fin ; ENDIF ;
% ;
% set up maximum number of sub-directories (packs of moves of 150 files each) ;
FOR j WITH (4 + (tell 2)) DO
d_moved := link d_base 'pack_' (string_prepad (string j) `0 2) '/' ;
IF flag_debug THEN
write ' ' ;
write link 'd_moved = ' d_moved ;
ENDIF ;
% move max number of files ;
FOR i WITH (tell 150) DO
IF (NOT isfault (text_line := readfile fin))
THEN
num_str := first str_cutBy_subStr text_line ' ' ;
fname := link 'N' (string_prepad num_str `0 4) '.pdf' ;
host (link 'mv "' d_base fname '" "' d_moved '"' ) ;
host 'sleep 0.1s' ;
ELSE EXIT null ;
ENDIF ;
ENDFOR ;
ENDFOR ;
close fin ;
}
#! 06Sep2021 for deletion : mv_files_listed_z IS z_pthL_moveEach
08********08
#] 17Oct2021 [path_isEmpty, path_exists]_tests
It turns out that path_exists was the problematic bash expression, so I converted it to use find
#] file_exists IS OP typer path - returns true if a [directory, path] exists (removed)
# 17Oct2020 test created
# fa := '/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
# fb := '/media/bill/SWAPPER/Website - raw/economics, markets/SP500/'
# file_exists '-f' fa
# file_exists "r fa
# file_exists '-d' fa
# file_exists "w fa
# loaddefs link d_Qtest 'fileops/file_ops- test.ndf'
IF flag_debug THEN write 'loading file_exists_test' ; ENDIF ;
file_exists_test IS
{ LOCAL t_name t_input t_standard t_result ;
NONLOCAL d_Qndfs ;
% ;
EACH write_testStr '#+-----+' (link 'str_to_unicodeList_test, ' timestamp) ;
% ;
IF flag_break THEN BREAK ; ENDIF ;
i_test := 0 ;
% ;
i_test := i_test + 1 ;
t_name := link '# file_exists_test example #' (string i_test) ;
t_input := link d_Qtest '/economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html' ;
t_standard := llol ;
t_result := EACH first ('-f' "r '-d' "w EACHLEFT file_exists t_input) ;
test_comment t_name t_input t_standard t_result ;
% ;
i_test := i_test + 1 ;
t_name := link '# file_exists_test example #' (string i_test) ;
t_input := link d_Qtest '/economics, markets/SP500/' ;
t_standard := ooll ;
t_result := EACH first ('-f' "r '-d' "w EACHLEFT file_exists t_input) ;
test_comment t_name t_input t_standard t_result ;
% ;
i_test := i_test + 1 ;
t_name := link '# file_exists_test example #' (string i_test) ;
t_input := link d_Qndfs 'Blue whale/economics, markets/SP500/' ;
t_standard := oooo ;
t_result := EACH first ('-f' "r '-d' "w EACHLEFT file_exists t_input) ;
test_comment t_name t_input t_standard t_result ;
}
IF flag_debug THEN write 'loading path_exists_tests' ; ENDIF ;
#] path_exists_tests IS -res ipsa loquitor
path_exists_tests IS
{ LOCAL n_cols n_rows paths_tbl test_tbl
p_0 p_1 p_2
r01 r02 r03 r04 r05 r06 r07
t01 t02 t03 t04 t05 t06 t07 ;
% ;
p_0 := link d_Qndfs '' ;
p_1 := link d_Qndfs 'strings.ndf' ;
p_2 := link d_Qndfs 'strngs.ndf' ;
% ;
t01 := 'path_exists "d_old p_0' ; r01 := execute t01 ;
t02 := 'path_exists "d_new p_0' ; r02 := execute t02 ;
t03 := 'path_exists "p_old p_0' ; r03 := execute t03 ;
t04 := 'path_exists "p_old p_1' ; r04 := execute t04 ;
t05 := 'path_exists "p_old p_2' ; r05 := execute t05 ;
t06 := 'path_exists "p_new p_1' ; r06 := execute t06 ;
t07 := 'path_exists "p_new p_2' ; r07 := execute t07 ;
% ;
paths_tbl :=
'path#' 'path'
'p_0' p_0
'p_1' p_1
'p_2' p_2
;
n_cols := 2 ; n_rows := floor ( (gage shape paths_tbl) / n_cols) ;
write (n_rows n_cols reshape paths_tbl) ;
% ;
test_tbl :=
'std=calc' 'test#' 'expression' 'result'
(= l r01) 't01' t01 r01
(= l r02) 't02' t02 r02
(= o r03) 't03' t03 r03
(= l r04) 't04' t04 r04
(= o r05) 't05' t05 r05
(= l r06) 't06' t06 r06
(= l r07) 't07' t07 r07
;
n_cols := 4 ; n_rows := floor ( (gage shape test_tbl) / n_cols) ;
write (n_rows n_cols reshape test_tbl) ;
}
# olde code
% && echo l >"' p_hostCmdRslt '" || echo o >"' p_hostCmdRslt '" ' ;
IF (= 'l' flag_exists) THEN result := l ;
ELSEIF (= 'o' flag_exists) THEN result := o ;
ELSE result := fault '?path_exists error : bash if error' ;
ENDIF ;
write 'flag_exists = ' flag_exists ' type ' (type flag_exists) ;
# olde code
IF flag_debug THEN write 'loading path_exists_tests' ; ENDIF ;
#] path_exists_tests IS -res ipsa loquitor
path_exists_tests IS
{ LOCAL n_cols n_rows paths_tbl test_tbl
p_0 p_1 p_2
r01 r02 r03 r04 r05 r06 r07
t01 t02 t03 t04 t05 t06 t07 ;
% ;
p_0 := link d_Qndfs '' ;
p_1 := link d_Qndfs 'strings.ndf' ;
p_2 := link d_Qndfs 'strngs.ndf' ;
% ;
t01 := 'path_exists "d_old p_0' ; r01 := execute t01 ;
t02 := 'path_exists "d_new p_0' ; r02 := execute t02 ;
t03 := 'path_exists "p_old p_0' ; r03 := execute t03 ;
t04 := 'path_exists "p_old p_1' ; r04 := execute t04 ;
t05 := 'path_exists "p_old p_2' ; r05 := execute t05 ;
t06 := 'path_exists "p_new p_1' ; r06 := execute t06 ;
t07 := 'path_exists "p_new p_2' ; r07 := execute t07 ;
% ;
paths_tbl :=
'path#' 'path'
'p_0' p_0
'p_1' p_1
'p_2' p_2
;
n_cols := 2 ; n_rows := floor ( (gage shape paths_tbl) / n_cols) ;
write (n_rows n_cols reshape paths_tbl) ;
% ;
test_tbl :=
'std=calc' 'test#' 'expression' 'result'
(= l r01) 't01' t01 r01
(= l r02) 't02' t02 r02
(= o r03) 't03' t03 r03
(= l r04) 't04' t04 r04
(= o r05) 't05' t05 r05
(= l r06) 't06' t06 r06
(= l r07) 't07' t07 r07
;
n_cols := 4 ; n_rows := floor ( (gage shape test_tbl) / n_cols) ;
write (n_rows n_cols reshape test_tbl) ;
}
08********08
#] 15Oct2021 path_isEmpty IS OP path - returns l if true and o if false
# path_isEmpty IS OP path - returns l if true and o if false
# Warning : if a path does not exist, then as per Linux convention, this returnss o
# (non-existent files aren't empty!)
# 15Oct2021 initial
16Oct2021 final version :
path_isEmpty IS OP path
{ IF (AND (path_exists "p_old path) (= 0 (path_size path))) THEN l ELSE o ENDIF }
# d_testy := link d_Qtest 'fileops/[test, std, result] files/' ;
# path
# path_isEmpty (link d_testy 'test- empty file.txt')
#] How to check if a file is empty in Bash? stackoverflow ...
https://stackoverflow.com/questions/9964823/how-to-check-if-a-file-is-empty-in-bash
[[ -f filename && ! -s filename ]] && echo "filename exists and is empty"
answered May 14 '19 at 13:25, Radek 'Goblin' Pieczonka
I could not get [[...]] to work when passed to Linux!!???
$ [[ -f "/home/bill/.selected_editor" && ! -s "/home/bill/.selected_editor" ]] && echo "l" || echo "o"
l
qnial> path_isEmpty (link d_testy 'test- empty file.txt')
cmd =
[[ -f '/media/bill/Dell2/Website - raw/Qnial/code develop_test/fileops/[test, std, result] files/test- empty file.txt' && ! -s '/media/bill/Dell2/Website - raw/Qnial/code develop_test/fileops/[test, std, result] files/test- empty file.txt' ]] && echo l || echo o
sh: 1: [[: not found
o
+-----+
chicken scratches
# typer := "p_old
# path := link d_Qndfs 'strings.ndf'
# path_exists typer path
# paty := path
# typet := '-f'
# a := link '[ ' typet ' "' paty '" ] && echo l >"' p_hostCmdRslt '" || echo o >"' p_hostCmdRslt '" '
# boolean first host_result a
+-----+
# olde code
% 05Jun2021 NYET? : result should be an error if paty is a problem? ;
IF noError THEN
result := boolean first host_result
(link '[ ' typet ' "' paty '" ] && echo l >"' p_hostCmdRslt
'" || echo o >"' p_hostCmdRslt '" '
) ;
ENDIF ;
# cart (4 reshape [d_temp link 'path_getOptrExpr_declDescLines - ']) ('optrDecl' 'optrDesc' 'exprDecl' 'exprDesc') (4 reshape ['.txt'])
# mix (4 reshape [d_temp link 'path_getOptrExpr_declDescLines - ']) ('optrDecl' 'optrDesc' 'exprDecl' 'exprDesc') (4 reshape ['.txt'])
# EACH link cart [d_temp link 'path_getOptrExpr_declDescLines - '] ('optrDecl' 'optrDesc' 'exprDecl' 'exprDesc') ['.txt']
# $ find "$d_Qndfs" -atime -90 -type f -name "*.ndf" | grep --invert-match 'z_Archive' | grep --invert-match 'z_Old' | tr \\n \\0 | xargs -0 -ILINE grep --with-filename --line-number "path_isEmpty" LINE
| sed 's/^\(.*\):[0-9]\+:.*/\1/' | sort -u
# $ echo "" >"$d_temp""path_isempty test.txt"
# path_isEmpty '/home/bill/.gksu.lock'
# path_isEmpty '/home/bill/.selected_editor'
# $ [[ -f "/home/bill/.selected_editor" && ! -s "/home/bill/.selected_editor" ]] && echo "l" || echo "o"
# https://stackoverflow.com/questions/9964823/how-to-check-if-a-file-is-empty-in-bash
edited Jun 15 at 9:38, Niklas R.
asked Apr 1 '12 at 13:41, Mich
[[ -f filename && ! -s filename ]] && echo "filename exists and is empty"
answered May 14 '19 at 13:25, Radek 'Goblin' Pieczonka
# $ d_testy="$d_test"'fileops/[test, std, result] files/'
# $ [[ -f "/home/bill/.selected_editor" && ! -s "/home/bill/.selected_editor" ]] && echo "l" || echo "o"
l
# $ [ -f "/home/bill/.selected_editor" && ! -s "/home/bill/.selected_editor" ] && echo "l" || echo "o"
bash: [: missing `]'
o
# olde code
cmd := link ' [[ -f "' path '" && ! -s "' path '" ]] && echo "l" || echo "o" ' ;
write cmd ;
cmd := link '[[ -f ' chr_apo path chr_apo ' && ! -s ' chr_apo path chr_apo ' ]] && echo l ||
qnial> path_isEmpty (link d_testy 'test- empty file.txt')
(( -f '/media/bill/Dell2/Website - raw/Qnial/code develop_test/fileops/[test, std, result] files/test- empty file.txt' && ! -s '/media/bill/Dell2/Website - raw/Qnial/code develop_test/fileops/[test, std, result] files/test- empty file.txt' )) && echo l || echo o
sh: 1: -f: not found
o
qnial> path_isEmpty (link d_testy 'test- empty file.txt')
( -f '/media/bill/Dell2/Website - raw/Qnial/code develop_test/fileops/[test, std, result] files/test- empty file.txt' && ! -s '/media/bill/Dell2/Website - raw/Qnial/code develop_test/fileops/[test, std, result] files/test- empty file.txt' ) && echo l || echo o
sh: 1: -f: not found
o
path_isEmpty IS OP path
{ write link 'path exists = ' (string (path_exists "p_old path)) ;
write link 'path size = ' (string (path_size path)) ;
IF (AND (path_exists "p_old path) (= 0 (path_size path))) THEN l ELSE o ENDIF }
08********08
#] 01Oct2021 pathL_replace_wordL[, _test, _test_output]
01Oct2021 initial
03Oct2021 added to:
"$d_Qroot""help - [develop, debug, error list, etc]/0_QNial lessons.txt"
section :
Lesson - Nested array (nestAry) transformers - can I get there?
Non-nested arrays (noNestAry) should be fine as standard [EACH, TWIG, LEAF, etc] should work.
Also, is there a problem with the way that TWIG works? I need my own TWAG transformer.
Questions :
Do you really want to do "first [str, ary] cart [L,A]" if some of your work focuses on single [chr, str] replacements?
If you force usage of "[,no]nestAry" [optr, tranformer]s, will it help raise your practicing meta-level of abstract [think, programm]ing? This could be the greatest benefit, irrespective of code efficiency?
+-----+
olde code
IF flag_debug THEN write 'loading pathL_replace_wordL' ; ENDIF ;
# pathL_replace_wordL IS OP pathL wordOldNewL flag_word flag_backup - change [str, word]L each pathL
# uses sed, therefore cannot simply handle sed restricted : search $.*/[\]^ replace &\/\n
pathL_replace_wordL IS OP pathL wordOldNewL flag_word flag_backup
{ LOCAL ptmp pathList wordOption ;
% ;
noerror := l ;
p_grep := link d_temp 'path_replace_wordL greps.txt' ;
ptmp := link d_temp 'path_replace_wordL temp.txt' ;
% ;
% test for illegal characters in wordOldNewL ;
% if I use `| instead of `/ as sed_separator, will that work (eg directories)? ;
IF (OR ('$.*/[]^' EACHLEFT = (EACH first wordOldNewL))
('&/' EACHLEFT = (EACH first wordOldNewL))
) THEN
noerror := o ;
write '?pathL_replace_wordL error: $.*/[]^ or &/ chrs in strOldNewL' ;
ENDIF ;
% ;
IF noerror THEN
FOR pinn WITH pathL DO
IF (NOT path_exists "p_old pinn)
THEN EACH write
'?pathL_replace_wordL, file unknown : '
pinn
'' ;
ELSE
IF flag_backup THEN EACH path_backupDatedTo_zArchive pathL ; ENDIF ;
IF flag_word THEN wordOption := '\b' ; ELSE wordOption := '' ; ENDIF ;
sed_expr1 := (link 's|' wordOption) EACHRIGHT link (EACH first wordOldNewL) ;
sed_expr2 := sed_expr1 EACHLEFT link (link wordOption '|') ;
% 11Sep2021 will the semi-colon cause a problem within the string? (shouldn't) ;
sed_expr3 := sed_expr2 EACHBOTH link ((EACH second wordOldNewL) EACHLEFT link '|;') ;
% remove the final trailing semi-colon ;
sed_expr4 := front link sed_expr3 ;
cmd := link 'sed "' sed_expr4 '" "' pinn '" >"' ptmp '" ' ;
write cmd ;
host cmd ;
host link 'mv "' ptmp '" "' pinn '"' ;
ENDIF ;
ENDFOR ;
ENDIF ;
}
# olde code
strL_cutBy_chrL IS OP strL chrL
{ LOCAL argAry boolAry boolLL ;
IF flag_break THEN BREAK ; ENDIF ;
boolTab := chrL EACHLEFT EACHRIGHT EACHRIGHT = strL ;
boolAry := EACH (solitary first) NOT OR EACH EACH cols boolTab ;
argAry1 := boolAry EACHBOTH pair (EACH solitary strL) ;
argAry := EACH EACH first argAry1 ;
EACH booleanL_cut_list argAry
}
08********08
#] 01Oct2021 pZarchive_restore IS OP path - restore most recent path_backupDatedTo_zArchive of a file,
link d_Qndfs 'Qndfs optr symbol changes.ndf':
# pZarchive_restore IS OP path - restore most recent path_backupDatedTo_zArchive of a file,
# retain the backup
did't I already do an earlier version?
NYET - only :
dirFlat_restoreTo_pathL IS OP flag_fnamesDated d_backup pathList -
restore dated paths from a FLAT d_backup, created with pathList_backupTo_dirFlat
>> might still come in handy... copied it to Working file link d_Qndfs 'Qndfs optr symbol changes.ndf'
will start from that
Also use code from :
# path_backupDatedTo_zArchive IS OP path - SAFE WORKING backup file to a z_Archive dated today (create?)
# zArchive is [specific, with respect] to each path, it is not an overall zArchive
# Notice that there is no error output (write) - that must be handeled by the calling optr!
from link d_Qndfs 'Qndfs optr symbol changes.ndf': example of a fileops test
# pathL_replace_wordL_test_output IS OP i_test wordOldNewL pstdPinnL -
# uses common template to execute & ouput diff to log file, for tests flag_backup is false
# 29Sep2021 initial, adapted from str_replaceIn_path_output
# see link d_Qtest 'file_ops- test.ndf'
# find recently accessed files containing a stringsort unique paths
# "-atime -90" means any file accessed less than 90 days ago
# $ find "$d_Qndfs" -atime -90 -type f -name "*.ndf" | grep --invert-match 'z_Archive' | grep --invert-match 'z_Old' | tr \\n \\0 | xargs -0 -ILINE grep --with-filename --line-number "_test" LINE | sed 's/^\(.*\):[0-9]\+:.*/\1/' | sort -u
>> OK, still lists MANY files!!
PROBLEM - path_backupDatedTo_zArchive doesn't work!
STR_CUTBY_CHAR - may be the problem
>> its my QNial timestamp_YYMMDD_HMS!!! returns error
(month_number Month) (day_2digit day)
find_Howell - fails now
find_Howell IS ary_find1st_subAry
arg order reversed now!
>> OK, fixed for now
Re-try : pZarchive_restore
qnial> path_backupDatedTo_zArchive (link d_Qroot 'code develop_test/fileops/test- file_ops.ndf')
>> good!!
Re-try pZarchive_restore_test (only one test at present)
>> looks good!!
Try all 3 pZarchive_restore_test
>> Good, works well
08********08
#] 27Oct2020 after [restructure, rename]ing of d_web[Raw, Site] :
webSite_list_htmlFiles
qnial> bye
qnial> loaddefs link d_Qndfs 'Website updates.ndf'
08********08
#] 27Oct2020 depther_global
qnial> webSite_convert
>> why does webSite_convert touch menuHeadFoots?
qnial> webSite_update
>> played around with pinn_writeExecute_pout, still doesn't work
>> check failures - original html of
Try :
depther := (gage shape (array_findAll_subArray `/ subDir)) ;
IF (1 < depther) THEN backtrack := link (depther reshape (solitary '../')) ;
ELSEIF (1 = depther) THEN backtrack := './' ;
ELSEIF (0 = depther) THEN backtrack := '' ;
>> Nyet - 1st directory down not working (depther = 1?)
>> put in write depther to track!
qnial> webSite_update
/media/bill/SWAPPER/Website - raw/Bill Howells videos/160901 Big Data, Deep Learning, and Safety/0_Big Data, Deep Learning, and Safety.html
depther = 2
depther = 1
depther = 1
depther = 1
depther = 1
depther = 1
>> Ah hah! depther recursion assigns wrong menuHeadFoot depthers!
>> Always = 1
Create depther_global and use for menus?
added depther_global, see 'file_ops.ndf'
[index.html, 'page ...'] had 'depther = 1', eg :
/media/bill/SWAPPER/Website - raw/page blogs.html
depther = 0
depther = 0
depther = 0
depther = 0
depther = 0
depther = 1
Also :
/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html
depther = 3
depther = 3
depther = 3
depther = 1
OOPSS! typo with
'fin Fotter'
Retry :
>> OK, all seem to be consistent, but did it work?
Home all work
Neural Nets none work (blue font in manu)
Projects all work
Software programming & code all work
Professional & Resume none work (blue font in manu)
Publications & reports all work
Howell-produced videos none work (blue font in manu)
Blogs all work
Cool stuff n/a - just lists directory content
Crazy themes and stories all work
Hosted sub-sites all work
Neil Howell's Art none work (blue font in manu)
>> same problem
Go back and adjust pinn_writeExecute_pout, see 'file_ops.ndf'
Find words (symbols) -w = word, -i = case insensitive
# find "$d_Qndfs" -maxdepth 4 -name "*.ndf" | tr \\n \\0 | xargs -0 -IFILE grep -w -i --with-filename --line-number 'depth_webWork' "FILE"
>> not used except for 'Website header.ndf', so I deleted
Now to add depther_global to 'file_ops.ndf'
IF (OR ('Menu' 'fin Head' 'fin Fotoer' 'fin footer' EACHLEFT subStr_in_str pinn))
THEN depther := depther_global ;
ELSE depther := (gage shape (array_findAll_subArray `/ subDir)) ;
depther_global := depther ;
ENDIF ;
Go back and adjust pinn_writeExecute_pout :
Change :
+.....+
depther := (gage shape (array_findAll_subArray `/ subDir)) ;
IF (1 < depther) THEN backtrack := link (depther reshape (solitary '../')) ;
ELSEIF (1 = depther) THEN backtrack := './' ;
ELSEIF (0 = depther) THEN backtrack := '' ;
+.....+
To :
+.....+
IF (OR ('Menu' 'fin Head' 'fin Footer' 'fin footer' EACHLEFT subStr_in_str pinn))
THEN depther := depther_global ;
ELSE depther := (gage shape (array_findAll_subArray `/ subDir)) + 1 ;
depther_global := depther ;
ENDIF ;
write link 'depther = ' (string depther) ;
backtrack := link (depther reshape (solitary '../')) ;
+.....+
qnial> lq_fileops ; webSite_convert ; webSite_update
>> Nothing works now
Change :
+.....+
IF (OR ('Menu' 'fin Head' 'fin Footer' 'fin footer' EACHLEFT subStr_in_str pinn))
THEN depther := depther_global ;
ELSE depther := (gage shape (array_findAll_subArray `/ subDir)) + 1 ;
depther_global := depther ;
ENDIF ;
write link 'depther = ' (string depther) ;
backtrack := link (depther reshape (solitary '../')) ;
+.....+
To :
+.....+
IF (OR ('Menu' 'fin Head' 'fin Footer' 'fin footer' EACHLEFT subStr_in_str pinn))
THEN depther := depther_global ;
ELSE depther := (gage shape (array_findAll_subArray `/ subDir)) ;
depther_global := depther ;
ENDIF ;
write link 'depther = ' (string depther) ;
backtrack := link (depther reshape (solitary '../')) ;
+.....+
>> YAYY!!! all main menu now works
>> sub-menus don't always work
submenus :
Home n/a
Neural Nets none work (blue font in manu)
Projects most work, but not [MindCode, Lucas, Puetz, COVID-19]
Software programming & code none work
Professional & Resume Resume works, not Education
Publications & reports n/a
Howell-produced videos none work (blue font in manu)
Blogs all work
Cool stuff n/a - just lists directory content
Crazy themes and stories all work except deer
Hosted sub-sites all work except Wickson
Neil Howell's Art all work except Wickson
>> getting better, but still a path to go
Check out 'Menu neural nets.html' sub-menus
Examples :
MindCode |
Computational neuro-genetic models |
>> remove extraneous 'Neural nets/'
Neural Nets
should work?
Computational neuro-genetic modelling.html - where is this?
Projects
One problem is that many web-directories were moved to new sub-directories of 'Projects - mini/'
Software programming & code
Yikes - I didn't even create a subdirectory? - maybe in separate directories
I made a new directory, and moved [Qnial, bin, LibreOffice macros, System_maintenance] into it
I modified 'Menu software programming.html' for this
Revamped d_webSite too
Professional & Resume
I have no idea why education.html doesn't work?
Publications & reports
n/a
Howell-produced videos
ALL had duplicaste sub-directories!!
Blogs
n/a for now as all work
Cool stuff
n/a
Crazy themes and stories
deer - corrected file name
Hosted sub-sites
Neil Howell's Art
I can't see anything wrong with 'Wickson website/Steven Wickson.html' ??
08********08
#] 26Oct2020 pinn_writeExecute_pout
convert pinn_writeExecute_pout to use 'stdTmp' when provided, uses [standard, global] temp file output
reduces "inventing variables for nothing"
# 26Oct2020 added pout to call arguments -
# as per setup.ndf :
stdInn := 0 for readfile (type input at terminal)
stdOut := 1 for output (to terminal)
stdTmp := 'stdTmp' ;
# 26Oct2020 normally, pout is unknown when calling pinn_writeExecute_pout. Use either [null, 'stdTmp']
$ find "$d_Qndfs" -maxdepth 2 -type f -name "*\.ndf" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "pinn_writeExecute_pout" "FILE"
Also - modify path_exists to accomodate [stdInn, stdOut, stdTmp]
>> stdTmp is OK - path_exists doesn't need to be modified.
Find words (symbols) -w = word, -i = case insensitive
# find "$d_Qndfs" -maxdepth 4 -name "*.ndf" | tr \\n \\0 | xargs -0 -IFILE grep -w -i --with-filename --line-number 'depth_webWork' "FILE"
>> not used except for 'Website header.ndf', so I deleted
Now to add depther_global to
IF (OR ('Menu' 'fin Head' 'fin Fotoer' 'fin footer' EACHLEFT subStr_in_str pinn))
THEN depther := depther_global ;
ELSE depther := (gage shape (array_findAll_subArray `/ subDir)) ;
depther_global := depther ;
ENDIF ;
+-----+
OK, now to change the "embedded executable" path_insertIn_fHand :
Change in webPage_convertEncoding :
+-----+
[#!: path_insertIn_fHand (link d_webWork 'Menu.html') fout
+-----+
To :
+-----+
[#!: pinn_writeExecute_pout (link d_webWork 'Menu.html') stdTmp d_webRaw d_webSite ; path_insertIn_fHand stdTmp fout ;
+-----+
08********08
#] 25Oct2020 pinn_writeExecute_pout IS OP path d_wrtRoot - update to p_temp using embedded executable coding
pinn_writeExecute_pout - change and test
key change :
dir := str_extractPast_strFront (path_extract_dir pinn) d_inn ;
pout := link d_out dir (path_extract_fname pinn) ;
depther is wrong -> one too few
+-----+
[save, reduce] cart [test, comment]s etc :
# pinn_writeExecute_pout IS OP path d_wrtRoot - update to p_temp using embedded executable coding
# (eg webPage)
# 13Oct2020 new ':&data-insert &:' section is [incomplete, untested]
# automatically adapt menu inserts to the directory depther of the webpage ;
# This allows the same basic fileName to be used in the code, avoiding the [headache, errors] of manual changes
# All [file-inserts, menues] reference files via paths starting from webRoot (simple)
# WARNING : only ONE string between quotes is allowed per line, or erros and a mess will result!
IF flag_break THEN BREAK ; ENDIF ;
# [variables, operators, etc] must be defined in [globals, path]
# backtrack is available to maintain relative addressing for projects, so directories can be moved around etc
# strings_between - not all will be simple text substitutes.
# 18Oct2020 initial
# IF not, return null so web-page not corrupted?
IF flag_break THEN BREAK ; ENDIF ;
pinn_writeExecute_pout IS OP pinn d_inn d_out
{ LOCAL dir base extn backtrack depther finn fout shaper
indicesMidls line midlNews pout strList ;
IF flag_break THEN BREAK ; ENDIF ;
dir := str_extractPast_strFront (path_extract_dir pinn) d_inn ;
pout := link d_out dir (path_extract_fname pinn) ;
IF (NOT AND (EACH path_exists ("r pinn) ("w pout)))
THEN EACH write
'?pinn_writeExecute_pout file unknown error, OR [pinn, pout] : '
pinn
pout
'' ;
ELSE
depther := (gage shape (array_findAll_subArray `/ dir)) + 1 ;
backtrack := link (depther reshape (solitary '../')) ;
finn := open pinn "r ;
fout := open pout "w ;
WHILE (NOT isfault (line := readfile finn)) DO
IF (= '[#!: ' (5 take line))
THEN execute (5 drop line) ;
ELSEIF (AND ('[#=; ' ' ;=#]' EACHLEFT in_string line)) THEN
indicesMidls strList := str_splitLftRgtTo_indices_StrList '[#=; ' ' ;=#]' line ;
IF (~= null indicesMidls) THEN
midlNews := strList#indicesMidls ;
midlNews := EACH execute midlNews ;
strList#indicesMidls := midlNews ;
shaper := gage shape indicesMidls ;
strList#(indicesMidls - 1) := shaper reshape (solitary null) ;
strList#(indicesMidls + 1) := shaper reshape (solitary null) ;
ENDIF ;
writefile fout (link strList) ;
ELSE
writefile fout line ;
ENDIF ;
ENDWHILE ;
EACH close finn fout ;
ENDIF ;
pout
}
# pinn_writeExecute_pout (link d_Qtest 'test- Menu.html') 'economics, markets/SP500/multi-fractal/'
# pinn_writeExecute_pout (link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html') 'economics, markets/SP500/multi-fractal/'
# qnial> path_backupDated_retain (link d_webRaw 'economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html')
# pinn_writeExecute_pout (link d_webRaw 'economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html') 'economics, markets/SP500/multi-fractal/'
# str_extractPast_strFront (link d_webRaw 'Pandemics, health, and the Sun/influenza/Howell - influenza virus.html') d_webRaw
# host 'rm "$d_temp""temp- pinn_writeExecute_pout.txt.*"'
# EACH execute '5' (link 'pass ' chr_apo 'hello world' chr_apo)
# liner := ' '
# '[#=; ' ' ;=#]' EACHLEFT in_string liner
# str_splitLftRgtTo_indices_StrList '' liner
# str_splitLftRgtTo_indices_StrList '[#=; ' ' ;=#]' liner
# strList#((2 - 1) (2 + 1)) := null null
# strList#(2 + 2) := null
# strList#(2 + 2) := solitary null
# (gage shape indicesMidls) reshape (solitary null)
# Common executables, examples :
backtracks the directory of an href to webRoot, depends on the number of directories down of webPage
[#!: path_insertIn_fHand (link d_head "fin Head_two.html") ;
inserts "fin Head_two.html" into fHand at the line it is encountered
[#!: path_insertIn_fHand (path_processExecutes (link d_head "Menu.html" ) depth_webWork) ;
processes "Menu.html" then inserts the processed file into fHand
# old code
% provide a maximum depth for many different path_insertIn_fHand ;
% This is enough for [fin Head_one.html, fin Head_two.html] and 4 menu levels ;
% but not enough for run-away file errors ;
% later, after testing and some time of using this system, I can delete the p_temps ;
FOR i WITH tell 40 DO
IF (path_exists "r p_temp)
THEN p_temp := link p_temp (string i) ;
ELSE fout := open p_temp "w ;
ENDIF ;
ENDFOR ;
08********08
#] new variant of [path_backupDated, path_backupDated_delete]
24Oct2020
For massive changes in [deep, broad] directories, it is impractical to backup each file in it's own directory. Afterwards, manual cleanout of directories would not happen. So the approach now is to save the file, with its original name, in a [flat, dated] directory whose "base" name is provided. The date will be of the type YYMMDD so that multiple passes on the same day simply overwrite previous iterations.
PROBLEM - Often, differnt files in different directories have tthe same name. At present, this operator overwrites earlier backups.
In the future - special naming (eg part of dirName) to avoid this, with checks.
Actually, you simply supply the directory, no matter the [dating, etc] - just put what you want.
08********08
#] path_exists IS OP typer path - returns true if a [directory, path] exists, or can be created
17Oct2020
search d_Qroot for file_exists
$ find "$d_Qroot" -maxdepth 4 -type f -name "*" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "file_exists" "FILE" | grep '||'
$ find "$d_Qroot" -maxdepth 4 -type f -name "*" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "&&" "FILE"
#| grep '||'
search "Linux bash and how do I test if a file exists"
https://devconnected.com/how-to-check-if-file-or-directory-exists-in-bash/
How To Check If File or Directory Exists in Bash
written by schkn
# file_exists "r '/media/bill/ramdisk/string_in results.txt'
# file_exists "w '/media/bill/ramdisk/'
# file_exists '-f' '/media/bill/ramdisk/string_in results.txt'
# file_exists '-d' '/media/bill/ramdisk/'
# EACH file_exists ("r fa) ("w fb))
# EACH file_exists ("r p_sort) ("w p_bad) ("w p_OK))
qnial> file_exists '-f' '/media/bill/ramdisk/string_in results.txt'
sh: 1: [[: not found
?address
$ fa='/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
$ fb='/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/'
$ [[ -f fa ]] && echo "This file exists!"
$ [ -f fa ] && echo "This file exists!"
# fa := '/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
# fb := '/media/bill/SWAPPER/Website - raw/economics, markets/SP500/'
# file_exists '-f' fa
# file_exists "r fa
# file_exists '-d' fa
# file_exists "w fa
# can't get "normal" blog appraoch to work in bash, revert to my old way
# first host_result '[ -f fa ] && echo l || echo o '
# first host_result '[ -d fb ] && echo l || echo o '
# gives error becaue fb is not a dir
# first host_result '[ "w fb ] && echo l || echo o '
# file_exists "r fa
# file_exists ( "w fb )
# EACH file_exists ( "r fa ) ( "w fb )
I screwed up far too many details. After many changes :
qnial> file_exists_test
#+-----+
str_to_unicodeList_test, Sat Oct 17 22:01:11 2020
# file_exists_test example #1 : OK - result matches standard
t_input, t_standard, t_result =
/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html
llol
llol
# file_exists_test example #1 : OK - result matches standard
t_input, t_standard, t_result =
/media/bill/SWAPPER/Website - raw/economics, markets/SP500/
ooll
ooll
# file_exists_test example #2 : OK - result matches standard
t_input, t_standard, t_result =
/media/bill/SWAPPER/Blue whale/economics, markets/SP500/
oooo
oooo
+-----+
>> OK looks good
# old code
{ LOCAL dir result ;
IF (OR (= "r typer) (= '-f' typer)) THEN
result := first host_result link '[ -f "' path '" ] && echo l || echo o ' ;
ELSEIF (= "w typer) THEN
dir := (last find_Howell `/ path) take path ;
result := first host_result link '[ -d "' dir '" ] && echo l || echo o ' ;
ELSEIF (= '-d' typer) THEN
result := first host_result link '[ -d "' path '" ] && echo l || echo o ' ;
ELSE result := fault '?file_exists error : unknown typer' ;
ENDIF ;
result
}
08********08
#] 22Apr2020 webPageList stuff
+-----+
# cat '/media/bill/ramdisk/webPageList.txt' | sed 's#:.*##'
#
FOR page WITH webPageList DO
% first check to see if the html file needs updating! ;
IF ( 0.1 < (- (EACH file_date (link dw_base page) (link ds_base page)))) THEN
write page ;
%update_webPage page ;
ENDIF ;
ENDFOR ;
}
webPageList := pNAME_read_list p_webPageList ;
% first check to see if the html file needs updating! ;
webPageList
}
webPageList := pNAME_read_list p_webPageList ;
FOR page WITH webPageList DO
write page
cmd := link 'grep -l ":@file-insert@:" "' page '" ' ;
host cmd ;
ENDFOR ;
}
cmd := link 'find "' webSite '" -maxdepth 3 -name "*.html" >"' p_webPageList '" ' ;
% find all html files with a ':@file-insert@:', grep -l outputs filname ;
cmd := link 'grep -l ":@file-insert@:" "' dw_base '" ' ;
host cmd ;
08********08
#] 22Apr2020 file_dateTimeStamp IS OP filePath - return the dateTimeStamp of the file (long integer)
22Apr2020 dateTimeStamps
# file_dateTimeStamp IS OP filePath - return the dateTimeStamp of the file (long integer)
# can be used for testing if a file is [younger, older] than a given dateTimeStamp
# result is a list, to facilitate later comparisons
file_dateTimeStamp IS OP filePath { tonumber first host_result (link 'stat -c %Y "' filePath '" ' ) }
# qnial> file_dateTimeStamp '/home/bill/PROJECTS/Qnial/MY_NDFS/translate c to nial.txt'
# qnial> < EACH file_dateTimeStamp '/home/bill/PROJECTS/Qnial/MY_NDFS/translate c to nial.txt' '/home/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf'
#
https://stackoverflow.com/questions/14842195/how-to-get-file-creation-date-time-in-bash-debian
How to get file creation date/time in Bash/Debian?
All the methods above are nice, but if you want to quickly get only last modify date, you can type:
ls -lit /path
with -t option you list all file in /path odered by last modify date.
answered Feb 21 '14 at 15:41, Luca Davanzo
$ man ls ->
# $ ls -lit "/home/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf"
# $ ls -l "/home/bill/PROJECTS/Qnial/MY_NDFS/translate c to nial.txt"
https://www.linuxquestions.org/questions/programming-9/get-file-modification-date-time-in-bash-script-163731/
# $ stat -c %Y "/home/bill/PROJECTS/Qnial/MY_NDFS/translate c to nial.txt"
1491787092
# $ date -r "/home/bill/PROJECTS/Qnial/MY_NDFS/translate c to nial.txt" +%F
2017-04-09
# $ stat -c %Y "/home/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf"
1587581049
# $ date -r "/home/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf" +%F
2020-04-22
$ find "$d_Qndfs" -maxdepth 1 -name "*" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "file_date" "FILE" | grep --invert-match "Is a directory" -
08********08
#] 05Apr2019 file_change_lines IS OP p_inn p_out linesInn linesOut - changes [p_inn linesInn]
#] to [p_out, linesOut] (text files)
# 05Apr2019 initial
#
testing - see '/media/bill/PROJECTS/Qnial/code develop_test/file_ops.ndf-code test.ndf'
08********08
#] 13Apr2018 file_exists stuff
# 13Apr2018 tests
# [ -f "/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf" ] && echo "File exist" || echo "File does not exist"
# qnial> file_exists '-f' '/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf'
# qnial> file_exists '-d' '/media/bill/PROJECTS/Qnial/MY_NDFS/'
# enddoc
|