Changed (got rid of chr_apo)
Also Change :
+.....+
!!linkError!!Lucas's Universal Force for electrodynamics, gravity, mass, etc
+.....+
To :
+.....+
[#=; backtrack ;=#]Lucas/
+.....+
!!linkError!!Howell - Are we ready for global cooling.pdf
# host link 'find "' d_webRaw '" -maxdepth 4 -type f -name "*.html" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "!!linkError!!Howell \- Are we ready for global cooling\.pdf" "FILE" | grep --invert-match "z_Old" | grep --invert-match "z_Archive" '
/media/bill/SWAPPER/Website - raw/page Publications & reports.html:62: Bill Howell "Are we ready for global cooling?" - A short presentation to Toastmasters – Dows Lake, Ottawa, 14Mar06. Needs corrections and comments! (some time later...)
Change :
+.....+
!!linkError!!Howell - Are we ready for global cooling.pdf
+.....+
To :
+.....+
[#=; backtrack ;=#]Howell - Are we ready for global cooling.pdf
+.....+
find 'Howell - Are we ready for global cooling.pdf' allFnamesSortedByFname) pick allFnamesSortedByFname
qnial> find_Howell 'Howell - Are we ready for global cooling.pdf' allFnamesSortedByFname) pick allPathsSortedByFname
?tokens left: FIND 'Howell - Are we ready for global cooling.pdf'
qnial> fnd `# 'my name # is Sue'
?undefined identifier: FND <***> `# 'my name # is Sue'
qnial> find `# 'my name # is Sue'
8
>> what's wrong with expression?
qnial> (find_Howell 'Howell - Are we ready for global cooling.pdf' allFnamesSortedByFname) pick allPathsSortedByFname
?address
>> It isn't in allPathsSortedByFname
>> Did I even write this? problematic, leave for now & add to ToDos
!!linkError!!International Neural Network Society.JPG
qnial> (find_Howell 'International Neural Network Society.JPG' allFnamesSortedByFname) pick allPathsSortedByFname
?address
!!linkError!!Neural nets/Conference guides/Author guide website/IEEE electronic Copyright form.html
>> This and others should be OK
do a mass remove of !!linkError!!
host link 'find "' d_webRaw '" -maxdepth 4 -type f -name "*.html" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "!!linkError!!" "FILE" | grep --invert-match "z_Old" | grep --invert-match "z_Archive" | sed "s/!!linkError!!//g" '
>> NUTS!! screen output only
>> NUTS! what a mess! it will re-generate the same problem!! A I have to change the original
In the replacement text: '&\/\n' ;
- Should have been :
host link 'find "' d_webRaw '" -maxdepth 4 -type f -name "*.html" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "!!linkError!!" "FILE" | grep --invert-match "z_Old" | grep --invert-match "z_Archive" | sed "s/!!linkError!!/[#=; backtrack ;=#]/g" '
I need to write a QNial optr to do this via p_temp
20:02 OK - now webSiteURLs_doAll
>> YIKES! huge delay to check urls, it looksike a failure of :
extern fails, OK] - no fails (eg 4040 code)? impossible as I didn't fix them!??
intern [fails = 1, OK = 0] impossible as intern list had [19, 343] at last sucessful run.
intern list on 86 bytes, 1 line
mailto, pgPosn weren't touched!
>> What went wrong - check a few files!
Try :
qnial> urls_check 'intern'
nah
interns webRaw don't have '[#=; backtrack ;=#]'!??? what a mess!!
>> SHIT - I goofed up with `;
Change :
+.....+
# str_changeIn_webRaw '!!linkError!!' '[#=; backtrack ;=#]'
+.....+
To :
+.....+
# str_changeIn_webRaw '!!linkError!!' '[\#=; backtrack ;=\#]'
+.....+
I have to restore last good version of d_webRaw
Double-shit - I stupidly dated the filesnot easy to restore!!!
Change path_backupDatedTo_dir
To path_backupTo_dir
Last set without dates : in '/media/bill/SWAPPER/Website - raw/z_Archive/201117 17h00m21s backups'
24************************24
19Nov2020 fix 'urls errors list.txt'
Examples :
!!linkError!!corona virus/#Corona virus models
>> put every link with `# into 'urls pgPosn list.txt'
>> It should already be doing that! :
ELSEIF (subStr_in_str '#' linker) THEN writefile fpos linker ;
>> changed to :
ELSEIF (in `# linker) THEN writefile fpos linker ;
!!linkError!!Neural nets/Conference guides/
>> subDir problem should now be OK
!!linkError!!Software programming & code/Qnial/MY_NDFS/MindCode/
>> link is wrong : moved dir, or [changed, wrong] fname
!!linkError!!International Neural Network Society.JPG
!!linkError!!LibreCalc bank account macro system.txt
!!linkError!!Civilisations and sun/Howell - Mega-Life, Mega-Death and the Sun II, towards a quasi-predictive model of the rise and fall of civilisations.pdf
!!linkError!!Civilisations and sun/Howell - Mega-Life, Mega-Death and the Sun, towards a quasi-predictive model of the rise and fall of civilisations.pdf
Some we [lost, not created] :
!!linkError!!LibreCalc bank account macro system.txt
Some [files, dir]s will NOT have been rsync'd : Qnial, etc etc
>finish an rsync program that I had started
I need [find-grep-sed]s to [locate, change] affected files, en mass and one by one as I work on them.
1. remove ALL !!linkError!! and re-run, that should reduce list by 20-40%
2. [find, fix] individual errors
24************************24
19Nov2020 fix urls smmary table
# a := tell 5
# b := reverse tell 10
qnial> (a (EACHLEFT EACHRIGHT subStr_in_str b)) sublist b
?first arg of sublist not boolean
qnial> (a EACHLEFT EACHRIGHT subStr_in_str b) sublist b
?first arg of sublist not boolean
qnial> (a EACHALL subStr_in_str b) sublist b
?first arg of sublist not boolean
qnial> (a ITERATE subStr_in_str b)
?first arg of sublist not boolean
qnial> EACH = (cart a b)
oooooooool
oooooooolo
oooooooloo
oooooolooo
oooooloooo
qnial> (rows (EACH = (cart a b))) EACHLEFT sublist b
+-+-+-+-+-+
|0|1|2|3|4|
+-+-+-+-+-+
# a := 'jim' 'nancy' 'john' 'betty' 'harry'
# b := 'fred' 'jim' 'harry' 'john' 'dan' 'nancy' 'betty' 'floyd'
qnial> (cols (EACH = (cart a b))) EACHLEFT sublist b
++------------05-----05-------------++-----------+------------+05-----05
||+----05-----05|+---+|05-----05-----+||+---05-----05|+----05-----05||+---+|
|||fred|nancy|||dan|||harry|floyd||||jim|betty|||john|harry||||dan||
||+----05-----05|+---+|05-----05-----+||+---05-----05|+----05-----05||+---+|
++------------05-----05-------------++-----------+------------+05-----05
qnial> (cols (EACH = (cart a b))) EACHLEFT sublist a
+05-----05-------+------++-------+-------++-------+
||+---+|05-----05|+----+||05-----05|05-----05||05-----05|
|||jim|||harry|||john||||nancy|||betty||||harry||
||+---+|05-----05|+----+||05-----05|05-----05||05-----05|
+05-----05-------+------++-------+-------++-------+
qnial> (rows (EACH = (cart a b))) EACHLEFT sublist a
+-------++-------++------+
|05-----05||05-----05||+----+|
||nancy||||betty||||john||
|05-----05||05-----05||+----+|
+-------++-------++------+
qnial> (rows (EACH = (cart a b))) EACHLEFT sublist b
05-----05-------+------+-------+-------------+
|+---+|05-----05|+----+|05-----05|05-----05-----+|
||jim|||nancy|||john|||betty|||harry|harry||
|+---+|05-----05|+----+|05-----05|05-----05-----+|
05-----05-------+------+-------+-------------+
# b := 'fred' 'jim' 'harry' 'john' 'dan' 'nancy' 'betty' 'floyd' 'eloise'
qnial> (rows (EACH = (cart a b))) EACHLEFT sublist b
05-----05-------+------+-------+-------+
|+---+|05-----05|+----+|05-----05|05-----05|
||jim|||nancy|||john|||betty|||harry||
|+---+|05-----05|+----+|05-----05|05-----05|
05-----05-------+------+-------+-------+
# b := 'fred' 'dan' 'betty' 'jim' 'harry' 'john' 'dan' 'nancy' 'betty' 'floyd' 'harry' 'eloise'
qnial> (rows (EACH = (cart a b))) EACHLEFT sublist b
05-----05-------+------+-------------+-------------+
|+---+|05-----05|+----+|05-----05-----+|05-----05-----+|
||jim|||nancy|||john|||betty|betty|||harry|harry||
|+---+|05-----05|+----+|05-----05-----+|05-----05-----+|
05-----05-------+------+-------------+-------------+
qnial> (rows (EACH subStr_in_str (cart a b))) EACHLEFT sublist b
05-----05-------+------+-------------+-------------+
|+---+|05-----05|+----+|05-----05-----+|05-----05-----+|
||jim|||nancy|||john|||betty|betty|||harry|harry||
|+---+|05-----05|+----+|05-----05-----+|05-----05-----+|
05-----05-------+------+-------------+-------------+
# b := 'fred' 'dan1' 'betty1' 'jim' 'harry1' 'john' 'dan2' 'nancy' 'betty2' 'floyd' 'harry2' 'eloise'
qnial> b := 'fred' 'dan1' 'betty1' 'jim' 'harry1' 'john' 'dan2' 'nancy' 'betty2' 'floyd' 'harry2' 'eloise'
+----+----+------+---+------+----+----05-----05------05-----05------+------+
|fred|dan1|betty1|jim|harry1|john|dan2|nancy|betty2|floyd|harry2|eloise|
+----+----+------+---+------+----+----05-----05------05-----05------+------+
qnial> (rows (EACH subStr_in_str (cart a b))) EACHLEFT sublist b 05-----05-------+------+---------------+---------------+
|+---+|05-----05|+----+|+------+------+|+------+------+|
||jim|||nancy|||john|||betty1|betty2|||harry1|harry2||
|+---+|05-----05|+----+|+------+------+|+------+------+|
05-----05-------+------+---------------+---------------+
05-----05
olde code
n_fails := sum (fails_i sublist counts) ;
n_OKKs := sum (unkns_i sublist counts) ;
n_unkns := sum (OKKs_i sublist counts) ;
# find "$d_Qndfs" -maxdepth 3 -type f -name "*.ndf" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number "merge_2_1" "FILE"
24************************24
18Nov2020
I adapted these. They work.
#] webURLs_extract IS - extract all link urls from a website [external, internal, menu, pagePosn]
#] urls_check IS OP linkType - create sublists of [internal,xternal] links classed as [fail, OK]
#] check internal with path_exists '-f', externals with curl
Backups gave errors (that's OK)
02--02
qnial> urls_check 'intern'
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/urls extern fails.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/urls extern OK.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/urls intern fails.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/urls intern OK.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
+--
qnial> urls_check 'extern'
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/urls extern fails.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/urls extern OK.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
02--02
HTTP/1.1 404 Not Found - was classified as OK for external link
I changed to :
IF (OR ('200' '300' '301' '302' EACHLEFT in_string curlHdr))
I need to make an easy list of code explanations
Adapt& run website_link_counts :
qnial> see "MERGE_2_1
merge_2_1 IS OPERATION A {
0 catenate ( A @ 0 A @ 1 ) }
qnial> website_link_counts
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/webSite linkType fnames.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
?path_backupDatedTo_dir file unknown error, OR [path dirBackup] :
/media/bill/SWAPPER/Website - raw/webWork files/webSite linkType counts table.txt
/media/bill/SWAPPER/Website - raw/z_Archive/201118 backups/
?f_html_reformat file unknown error : p_linkTypeL
Ouput file :
?invalid left arg in split ?invalid left arg in split
OK - the code mostly works, now to look at failures
subDirs - are failing,perhaps because of '-f' option for p_exists?
05-----05
remove olde code from urls_check :
allLinks := strList_readFrom_path p_webSiteURLlist ;
% construct path for each link ;
p_list p_clean p_sort := EACH execute
( EACH link (cart ((solitary 'p_') (solitary linkType) (solitary 'URL') ('s' '_clean' '_sort') ))
) ;
IF (NOT AND (EACHRIGHT file_exists ("r p_list) ("w p_clean)))
THEN write fault link '?urls_check file unknown error : p_list or d_clean' ;
ELSE
flst := open p_list "r ;
fcln := open p_clean "w ;
write (2 2 reshape flst p_list fcln p_clean) ;
WHILE (~= ??eof (line := readfile flst)) DO
%write line ;
% ;
IF (OR (= 'howell' linkType) (= 'extern' linkType)) THEN
% 08Oct2020 I can't think of any cleaning required - really just copy ;
% in the case of 'howell', the source files will have to be corrected to be internal links only ;
writefile fcln line ;
% ;
ELSE
% extract text of line ;
colons := findall `: line ;
p_path := (first colons) take line ;
IF (= 'pgPosn' linkType)
THEN d_base := p_path ;
ELSE d_base := ((last findall `/ p_path) + 1) take p_path ;
ENDIF ;
lineTxt := rest drop (second colons) line ;
lineTxt := (find `" lineTxt) take lineTxt ;
%write link d_base lineTxt ;
% crawl up path_link directory reversals, at the same time truncating path_src ;
WHILE (~= null lineTxt) DO
IF (= './' (2 take lineTxt)) THEN lineTxt := 2 drop lineTxt ; %d_base := d_base ;
ELSEIF (= '../' (3 take lineTxt)) THEN lineTxt := 3 drop lineTxt ; d_base := ((last front findall `/ d_base) + 1) take d_base ;
ELSE EXIT 'urls_internal_check' ;
ENDIF ;
ENDWHILE ;
writefile fcln (link d_base lineTxt) ;
% ;
ENDIF ;
ENDWHILE ;
EACH close flst fcln ;
host link 'sort -u "' p_clean '" >"' p_sort '" ' ;
host 'sleep 1s' ;
ENDIF ;
# olde code
IF (= 'pgPosn' linkType) THEN
p_link pagePosn := string_cut_by_char `# p_link ;
IF (NOT file_exists '-f' p_link)
THEN write fault link '?urls_check file unknown error : p_link' ;
ELSE ????????????????????
% check if pagePosition is setup ;
IF (= null (host_result link 'grep ' chr_apo '' chr_apo ' "' p_link '" '))
THEN writefile fbad p_link ;
ELSE writefile fOKK p_link ;
ENDIF ;
ENDIF ;
# old code
writefile fout (first host_result link 'wc -l "' (link d_webRaw fname) '" ') ;
% tbl_rows := link table_rows (host_result (link 'wc -l "' (link d_webRaw fname) '" '))) ;
write table_rows ;
% picture merge_2_1 tbl_linkCnt tbl_tots ;
#
p_htmlPathsSiteList htmlPathsSiteSortedByPath ;
cmd := link 'grep -E -i --with-filename --line-number ">"' p_webSiteURLlist '" ' ;
24************************24
17Nov2020 ALL webPages :
+----+
7. check all links with 'website urls.ndf' & grep !!linkError!!
see link d_Qndfs 'website urls notes.txt'
24************************24
17Nov2020 ALL webPages :
1. backups
qnial> pathList_backupTo_dir htmlPathsSortedByPath d_htmlBackup
>> OK
2. pathList_change_headFoot htmlPathsSortedByPath
>> This was already done.
>> Check - OK
3. Make sure that [data, optrs] are up-to-date
qnial> lq_fileops
qnial> loaddefs link d_Qtest 'Website updates- tests.ndf'
4. d_webRaw - update links, ensure [full, proper] subDir & backtrack
>> first time use!!!
qnial> webAllRawOrSite_update l "webPageRaw_update
>> seems OK, no faults
5. d_webSite execute embeds [menu, Head, Foot, body], provide proper relative links
>> first time use!!!
qnial> webAllRawOrSite_update l "webPageSite_update
6. check 5 random webPageSites for results :
check all main menu items
check all subMenu items, but for common subMenus, just do once
check footer - [GNU, Creative Commons] images,
List of webPages selected :
'page Software programming.html'
'Charvatova solar inertial motion & activity/_Charvatova - solar inertial motion & activity.html'
'Neil Howell/_Neil Howell.html'
'Professional & Resume/_Resumes, work experience.html'
'Solar modeling and forecasting/_Solar modeling & forecasting.html'
'Solar modeling and forecasting/_Solar modeling & forecasting.html' :
[Puetz, Randell Mills] links STILL fail!! : exasperating!,
>> these were NOT updated!!! files last changed 27Oct2020!?!?!?
d_webRaw update 14:41 ?? - this wasn't done either!!??
201117 16h25m37s backups - three backups since 16:09, this is the most recent
No use looking at the other webPageSites. For some reason, webAllRawOrSite_update doesn't work.
What am I missing?
Redo - this time do pathList_change_headFoot
OK pathList_change_headFoot htmlPathsSortedByPath
OK webAllRawOrSite_update l "webPageRaw_update
NO webAllRawOrSite_update l "webPageSite_update
>> so the last step above doesn't work. Why?
>> The processing for both was far too fast. Something's wrong
check for file updates beyond 16:50 :
NO webAllRawOrSite_update l "webPageRaw_update
>> however, there is a new timed backup dir
>> So why doesn't webAllRawOrSite_update work?
webAllRawOrSite_update
Change :
+.....+
apply optr_rawOrSite flag_backup webPage ;
+.....+
To :
+.....+
apply optr_rawOrSite (flag_backup webPage) ;
+.....+
Re-try
qnial> loaddefs link d_Qtest 'Website updates- tests.ndf'
qnial> webAllRawOrSite_update l "webPageRaw_update
>> OK, at least one webPage updated (several more also)
>> runs much slower (only takes 5-10s?)
qnial> webAllRawOrSite_update l "webPageSite_update
>> OK, at least one webPage updated (several more also)
>> runs much slower (only takes 5-10s?)
So that was the problem.
05-----05
Redo 6. check 5 random webPageSites for results :
check all main menu items
check all subMenu items, but for common subMenus, just do once
check footer - [GNU, Creative Commons] images,
List of webPages selected :
'page Software programming.html'
mainMenu 'page Publications & reports.html' gives "' reports.html " at top of web-page, but goes to right page
perhaps this is due to an apostrophe in the fname or whatever?
mainMenu '_Neil Howell.html' - images of paintings don't appear, but [GNU, Creative Commons] do
subMenu - QNial link doesn't go to my new web-page
footer - dir list and [GNU, Creative Commons] images : OK
'Charvatova solar inertial motion & activity/_Charvatova - solar inertial motion & activity.html'
mainMenu 'page Publications & reports.html' gives "' reports.html " at top of web-page
mainMenu '_Neil Howell.html' - images of paintings don't appear, but [GNU, Creative Commons] do
same as for previous webPage
subMenu Charvatova - gives "' activity.html " at top of web-page (like earlier case)
subMenu Projects
failed links - Randell Mills,
S&P500 1872-2020, 83y trend - goes to covid-19 webPage!!
COVID-19 - goes to S&P500
Cdn Solar Forecast - gives "' forecasting.html " at top of web-page (like earlier cases)
SAFIRE - electric sun experiment - add as well to [economics, markets] subMenu
footer - dir list and [GNU, Creative Commons] images : OK
'Neil Howell/_Neil Howell.html'
mainMenu 'page Publications & reports.html' gives "' reports.html " at top of web-page
mainMenu '_Neil Howell.html' - images of paintings don't appear, but [GNU, Creative Commons] do
subMenu Hosted sites - OK
footer - dir list and [GNU, Creative Commons] images : OK
'Professional & Resume/_Resumes, work experience.html'
mainMenu 'page Publications & reports.html' gives "' reports.html " at top of web-page
mainMenu '_Neil Howell.html' - images of paintings don't appear, but [GNU, Creative Commons] do
subMenu Professional - OK
footer - dir list and [GNU, Creative Commons] images : OK
'Solar modeling and forecasting/_Solar modeling & forecasting.html'
Cdn Solar Forecast - gives "' forecasting.html " at top of web-page (like earlier cases)
mainMenu 'page Publications & reports.html' gives "' reports.html " at top of web-page
mainMenu '_Neil Howell.html' - images of paintings don't appear, but [GNU, Creative Commons] do
I didn't recheck Projects subMenu
footer - dir list and [GNU, Creative Commons] images : OK
+----+
7. check all links with 'website urls.ndf' & grep !!linkError!!
24************************24
17Nov2020 New menus items :
05-----05
'Menu projects.html' - rearrange and add missing webPages:
Lies, Damned Lies
Pandemics (general link)
influenza
corona virus
suicide
economics & markets
SP500-Schiller vs T-bill
multifractal
Fiboci mirror
Fed Rese control zone
Problematic web-pages -are due to [move, dirChanges]
Randell Mills
Puetz & Borchardt
/media/bill/SWAPPER/Website - raw/Projects - mini/Puetz & Borchardt/Howell - comments on Puetz UWS, the greatest of cycles, human implications.odt
/media/bill/SWAPPER/Website - raw/Projects - mini/Randell Mills/Howell - review of Holverstott 2016 Randell Mills hydrino energy.pdf
05-----05
'Menu neural nets.html'
Neural nets main link
OK - leave the rest of the [subDir, fname] corrections for later, after the url checks
24************************24
17Nov2020 Test webPages : Post-backups, check all links!
13:43 Is str_splitLftRgtTo_midIndxs_StrList still a problem?
midIndxs lineList := str_splitLftRgtTo_midIndxs_StrList strLeft strRight liner ;
put '[#=; backtrack ;=#]' into 5 files :
'fin [Footer, footer [Neil Howell,Paul Vauhan,Steven Wickson,Steven Yaskell].html'
qnial> loaddefs link d_Qtest 'Website updates- tests.ndf'
qnial> webPageRaw_update_test l (do all as the files aren't being over-written)
qnial> webPageSite_update_test l
Oh shit!
'whole-line-embed footer.txt'
Change :
+.....+
[#!: path_insertIn_fHand (link d_webWork 'fin Footer.html') fout ;
+.....+
To :
+.....+
[#!: path_executeEmbedsInsertIn_fHand (link d_webRaw 'webWork files/fin Footer.html') phraseValueList ;
+.....+
qnial> webPageRaw_update_test l (do all as the files aren't being over-written)
qnial> webPageSite_update_test l
Can't just do that. I have to repeat pathList_change_headFoot
pathList_change_headFoot htmlPathsSortedByPath
webPageRaw_update_test : I don't think that I need to do this, as subDirs already put in?
qnial> webPageSite_update_test l
>> OK, [GNU, Creative Commons] images ap
>> BUT! stupid `; problem is back
pathList_change_headFoot - remove `; ???
Change :
+.....+
cmd := link 'cat "' p_temp3 '" | sed "s|\(.*\)\(.*\)|<TITLE> ' fname ' ; |" >"' p_temp4 '"' ;
+.....+
To :
+.....+
cmd := link 'cat "' p_temp3 '" | sed "s|\(.*\)\(.*\)|<TITLE> ' fname ' |" >"' p_temp4 '"' ;
+.....+
>> I thought that I had alrdone this - probably directly in the files?
qnial> pathList_change_headFoot htmlPathsSortedByPath
qnial> webPageSite_update_test l
>> I STILL get a '`
1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html ;
This appears in 'pathList_change_headFoot temp4.txt' :
Steven Wickson.html ;
>> OOPS! forgot qnial> lq_fileops
OK AWESOME! - basics seem to work well... (maybe older files will have problems?)
05-----05
12:57 With the 3-problem fixes below, test webPage[Raw, Site]s
qnial> lq_fileops
qnial> loaddefs link d_Qtest 'Website updates- tests.ndf'
qnial> webPageRaw_update_test l
qnial> webPageSite_update_test l
1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html
page Howell - blog.html
Howell - corona virus.html
_Lies, damned lies, and scientists.html
Howell - influenza virus.html
>> NUTS!!! I over-wrote the webPageRaws!!! (fuck-up)
fix 'webPageSite_update'
Change back to previous :
+.....+
host link 'cp "' p_temp '" "' webPage '"' ;
+.....+
To :
+.....+
host link 'cp "' p_temp '" "' p_webSite '"' ;
+.....+
Copy-back from backup dir
qnial> webPageSite_update_test l
>> no extraneous `;
>> test- files NOT stored on webSite (where are they stored?
>> failed [GNU, Creative Commons] images
Damned failed [GNU, Creative Commons] images
'1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html' :
Perhaps these have moved. Is p_allFileList up to date?
/media/bill/SWAPPER/Website - raw/Creative commons.png - correct subDir fname (both web[Raw, Site])
/media/bill/SWAPPER/Website - raw/gnu-head-mini.png - correct subDir fname (both web[Raw, Site])
/media/bill/SWAPPER/Website - raw/logo Bill OReilly.png
/media/bill/SWAPPER/Website - raw/logo Creative Commons.png
webPageRaw_update - check to make sure that '"' is used as StrRght, not '">'
THEN line := internalLinks_return_relativePath backtrack '> these are OK
Problematic web-pages -are due to [move, dirChanges]
Randell Mills
Puetz & Borchardt
08:51 (earlier)
05-----05
First run all webPageRaw_update_test with save flag_backup = l, check embeds
loaddefs link d_Qtest 'Website updates- tests.ndf'
webPageRaw_update_test l
>> oops - missing headFoot
fileops.ndf inserted :
d_webWork := link d_webRaw 'webWork files/' ;
>> now it looks good!
>> must have had this in global varSpace before...
05-----05
Now, run all webPageSite_update_test with save flag_backup = l, check links
qnial> webPageSite_update_test l
subDirfname = economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html
>> `; appears at top of webPage?
>> [menuHeadFoot, body]-links look good (except known problems links)
>> [GNU, Creative Commons] images don't appear
subDirfname = page Howell - blog.html
>> same as 1st, but GNU image does appear
subDirfname = Pandemics, health, and the Sun/corona virus/Howell - corona virus.html
>> same as 1st
subDirfname = Lies, Damned Lies, and Scientists/_Lies, damned lies, and scientists.html
>> same as 1st, also 'Problems with Science ' subMenu links don't work :
Lies, Damned Lies, and Scientists
GR turkey, QM fools paradise
subDirfname = Pandemics, health, and the Sun/influenza/Howell - influenza virus.html
>> same as 1st
05-----05
Fix the `; problem
Can't see where this comes from
check 'pathList_change_headFoot temp[1-5].txt'
temp1 - no
temp2 - no
temp3 - no
temp4 - no
So it must be the temp5 step
pathList_change_headFoot
Change :
+.....+
cmd := link 'cat "' p_temp4 '" | sed "s|\(\[#!: str_executeEmbeds (link d_webWork ' chr_apo 'Menu\)\(.*.html' chr_apo ')\)\(.*\)|\[#!: path_executeEmbedsInsertIn_fHand (link d_webWork ' chr_apo 'Menu\2 phraseValueList ; |" >"' p_temp5 '"' ;
+.....+
To :
+.....+
cmd := link 'cat "' p_temp4 '" | sed "s|\(\[#!: str_executeEmbeds (link d_webWork ' chr_apo 'Menu\)\(.*.html' chr_apo ')\)\(.*\)|\[#!: path_executeEmbedsInsertIn_fHand (link d_webWork ' chr_apo 'Menu\2 phraseValueList |" >"' p_temp5 '"' ;
+.....+
>> removed `;, but I can't see how this will help
>> problem is header, not the footer
whole-line-embed header.txt
[#!: full-line executeEmbeds, phraseValueList = (("fout fout)("backtrack backtrack))
[#!: path_insertIn_fHand (link d_webWork 'fin Head_one.html') fout ;
[#!: writefile fout ' path ' ;
[#!: path_insertIn_fHand (link d_webWork 'fin Head_two.html') fout ;
>> but this is similar to the footer, which doesn't have this problem...
[#!: full-line executeEmbeds, phraseValueList = (("fout fout)("backtrack backtrack))
[#!: path_insertIn_fHand (link d_webWork 'fin Footer.html') fout ;
>> So maybe the problem is from :
[#!: writefile fout ' path ' ;
>> except the `; comes BEFORE the menus
This is driving me nuts...
outputwebPageRaw : has ' QNial.html ; '
That`; will be a problem
'whole-line-embed header.txt'
Change :
+.....+
QNial.html ;
+.....+
To :
+.....+
QNial.html
+.....+
05-----05
Fix the [GNU, Creative Commons] images
'fin [Footer, footer [Neil Howell,Paul Vauhan,Steven Wickson,Steven Yaskell].html'
Change :
+.....+
+.....+
To :
+.....+
+.....+
>> Hopefully, that will do it.
05-----05
Redirect 'test- *' file creation to z_Archive
This was done even today! it's not an olde thing
Not in :
'fileops.ndf'
'Website updates- tests.ndf'
'Website updates.ndf'
'Website header.ndf'
'strings.ndf'
Where the heck is it?
check all backup optrs
path_backupDatedToSameDir IS OP path - backup dated version of a file in same directory
path_backupDatedTo_dir IS OP path dirBackup - backup dated version of a file to a specified FLAT dir
dirBackup_restoreTo_paths IS OP d_backup p_pathList - restore paths listed in a backup (FLAT) dir
path_backupDated_delete IS OP path - rename a file with date precursor
Maybe it's in MY_NDFS somewhere other than the ndfs above?
Oop the ORIGINAL filenames have 'test- '
problem is that they are saved to d_webRaw, not d_Qtest
Look in 'Website updates.ndf'
>> I can't see where the problem occurs... ???
webPageSite_update
Change :
+.....+
p_webSite := link d_webSite subDir fname ;
write link 'subDirfname = ' subDir fname ;
IF (path_exists '-f' p_temp)
THEN
host (link 'diff --width=85 "' p_webSite '" "' p_temp '" --suppress-common-lines | grep ' chr_apo '^>' chr_apo ' | sed ' chr_apo 's/^>\ //' chr_apo ' >>"' p_log '"') ;
host link 'echo "" >>"' p_log '"' ;
IF flag_backup THEN
host link 'mv "' p_temp '" "' p_webSite '"' ;
ENDIF ;
ELSE
host link 'echo ?webPageSite_update error, p_temp not created >>"' p_log '"' ;
ENDIF ;
+.....+
To :
+.....+
p_webSite := link d_webSite subDir fname ;
write link 'subDirfname = ' subDir fname ;
IF (path_exists '-f' p_temp)
THEN
host (link 'diff --width=85 "' p_webSite '" "' p_temp '" --suppress-common-lines | grep ' chr_apo '^>' chr_apo ' | sed ' chr_apo 's/^>\ //' chr_apo ' >>"' p_log '"') ;
host link 'echo "" >>"' p_log '"' ;
IF flag_backup THEN
host link 'mv "' p_temp '" "' webPage '"' ;
ENDIF ;
ELSE
host link 'echo ?webPageSite_update error, p_temp not created >>"' p_log '"' ;
ENDIF ;
+.....+
>> Hopefully, that will do it.
24************************24
16Nov2020 webPageSite_update_test - check actual change in webPageSite
Note: there STILL seems to be a subDir problem with menus?
Just run & check
qnial> webPageSite_update_test l
main menu - all links are OK
Projects : all are EXCEPT (as before)
Randell Mills- hydrinos
Puetz - The Greatest of cycles
bodyLinks : All seem OK (not many)
external : I noticed that this didn't work
Ben Davidson of Suspicious Observers
Try all 5 test webpages
'economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
'page Howell - blog.html'
'Pandemics, health, and the Sun/corona virus/Howell - corona virus.html'
'Lies, Damned Lies, and Scientists/_Lies, damned lies, and scientists.html'
'Pandemics, health, and the Sun/influenza/Howell - influenza virus.html'
First I have to update the whole-line-embeds.
>> OK, all tests look good
Backup all targeted webPages :
qnial> pathList_backupTo_dir htmlPathsSortedByPath (link d_webRaw 'z_Archive/')
cp: cannot stat '/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html': No such file or directory
cp: cannot stat '/media/bill/SWAPPER/Website - raw/Lies, Damned Lies, and Scientists/test- _Lies, damned lies, and scientists.html': No such file or directory
cp: cannot stat '/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/corona virus/test- Howell - corona virus.html': No such file or directory
cp: cannot stat '/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/influenza/test- Howell - influenza virus.html': No such file or directory
cp: cannot stat '/media/bill/SWAPPER/Website - raw/test- page Howell - blog.html': No such file or directory
>> I deleted the test- webPages
05-----05
qnial> pathList_change_headFoot htmlPathsSortedByPath
test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html
cat: '/media/bill/SWAPPER/Website - raw/economics, markets/SP500/multi-fractal/test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html': No such file or directory
test- _Lies, damned lies, and scientists.html
cat: '/media/bill/SWAPPER/Website - raw/Lies, Damned Lies, and Scientists/test- _Lies, damned lies, and scientists.html': No such file or directory
test- Howell - corona virus.html
cat: '/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/corona virus/test- Howell - corona virus.html': No such file or directory
test- Howell - influenza virus.html
cat: '/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/influenza/test- Howell - influenza virus.html': No such file or directory
test- page Howell - blog.html
cat: '/media/bill/SWAPPER/Website - raw/test- page Howell - blog.html': No such file or directory
>> only failures are noed above, basically only test files
>> - so all normal webPages were processed (very quickly!)
24************************24
16Nov2020 webPageSite_update, [str, path]_executeEmbeds
qnial> webPageSite_update_test o
headFoots OK
Menus : backtrack remains a string!?
Home
small or sleeping : |
bodyLinks : don't even appear
Break for brunch
qnial> webPageSite_update_test o
-------------------------------------------------------------
Break debug loop: enter debug commands, expressions or
type: resume to exit debug loop
executes the indicated debug command
current call stack :
webpagesite_update_test
webpagesite_update_output
webpagesite_update
str_executeembeds
path_executeembedsinsertin_fhand
path_executeembeds
str_executeembeds
-------------------------------------------------------------
-->[stepv] nextv
?.. IF ( not or ( isfault Midindxs ) ( = Null Midindxs ) ) THEN
Midlinks := EACH execute Midlinks ;
(...)
-->[nextv] str phraseValueList
+----------------------------------------------------------+----------------------------+
| |+------+-------------------+|
| ||fout 6|backtrack ?no_value||
| |+------+-------------------+|
+----------------------------------------------------------+----------------------------+
>> Oops, n value for backtrack - co
Change :
05-----05
path_executeEmbeds IS OP path phraseValueList
{ LOCAL backtrack finn fout line p_temp ;
% ;
% Create p_temp (str), and delete past versions so update failures will result in a diff error message ;
p_temp := executeEmbedsGet_pathTemp ;
IF (path_exists '-f' p_temp) THEN host link 'rm "' p_temp '"' ; ENDIF ;
% ;
finn := open path "r ;
fout := open p_temp "w ;
WHILE (NOT isfault (line := readfile finn)) DO
IF (OR ('[#!: ' '[#=; ' EACHLEFT subStr_in_str line))
THEN line := str_executeEmbeds line (("fout fout)("backtrack backtrack)) ;
ENDIF ;
writefile fout line ;
ENDWHILE ;
EACH close finn fout ;
p_temp
}
+.....+
To :
+.....+
path_executeEmbeds IS OP path phraseValueList
{ LOCAL backtrack finn fout line p_temp ;
% ;
% Create p_temp (str), and delete past versions so update failures will result in a diff error message ;
p_temp := executeEmbedsGet_pathTemp ;
IF (path_exists '-f' p_temp) THEN host link 'rm "' p_temp '"' ; ENDIF ;
% ;
finn := open path "r ;
fout := open p_temp "w ;
WHILE (NOT isfault (line := readfile finn)) DO
IF (OR ('[#!: ' '[#=; ' EACHLEFT subStr_in_str line))
THEN line := str_executeEmbeds line phraseValueList ;
ENDIF ;
writefile fout line ;
ENDWHILE ;
EACH close finn fout ;
p_temp
}
+.....+
Retry
qnial> webPageSite_update_test o
>> still:
backtrack rather than n*'../'
bodyLinks lack subDir!
str_executeEmbeds doesn't put the midLinks back into strList!!
I must have dropped it during copy-paste?
Change :
+.....+
midLinks := EACH execute midLinks ;
+.....+
To :
+.....+
strList#midIndxs := EACH execute midLinks ;
+.....+
Retry
qnial> webPageSite_update_test o
>> OK!! '../../' substituted for 'backtrack'
>> bodyLinks
05-----05
K.F. Tapping, R.G. Mathias, D.L. Surkan,
"
Pandemics and solar activity" . This is an unpublished, expanded paper of
the 2001 paper by the same authors, which is listed in the references below.
05-----05
becomes
05-----05
K.F. Tapping, R.G. Mathias, D.L. Surkan,
Pandemics and solar activity" . This is an unpublished, expanded paper of
the 2001 paper by the same authors, which is listed in the references below.
05-----05
>> '' subStr_in_str str) THEN BREAK ; ENDIF ;
>> this worked within str_executeEmbeds
-->[nextv]
"
subDirfname = Pandemics, health, and the Sun/influenza/Howell - influenza virus.html
>> so why isn't it in the output file?
In webPageSite_update :
05-----05
WHILE (NOT isfault (line := readfile finn)) DO
% first executeEmbeds if present, with (phrase values) pairList ;
IF (OR ('[#!: ' '[#=; ' EACHLEFT subStr_in_str line)) THEN
line := str_executeEmbeds line (("fout fout)("backtrack backtrack)) ;
ENDIF ;
% now process links ;
IF ('> This is BACKWARDS!! Change to :
05-----05
WHILE (NOT isfault (line := readfile finn)) DO
% process links ;
IF ('' subStr_in_str str) THEN BREAK ; ENDIF ;
Re-Try
Here is the problem : ?
05-----05
-->[nextv] line
"
-->[nextv]
?.. ( '[nextv]
"
?.. Line := str_executeembeds Line ( ( "fout Fout ) ( "backtrack
Backtrack ) )
-->[nextv]
"
05-----05
>> missing subDir!?, but at least the '../../' is written to the output now.
internalLinks_return_relativePath isn't putting in the subDir - why?
Move break to internalLinks_return_relativePath :
IF ('' subStr_in_str line) THEN BREAK ; ENDIF ;
Nuts! - elimiating line because of `#
05-----05
?.. ( or ( Midindxslines_bads EACHLEFT substr_in_str Linelist @ I ) )
-->[nextv]
?.. or ( Midindxslines_bads EACHLEFT substr_in_str Linelist @ I )
-->[nextv]
l
l
?.. Null
05-----05
Remove '#' from midIndxsLines_bads
Change :
+.....+
IF (OR (midIndxsLines_bads EACHLEFT subStr_in_str lineList@i)) THEN
+.....+
To :
+.....+
IF (OR (= `# (first lineList@i)) (OR (midIndxsLines_bads EACHLEFT subStr_in_str lineList@i))) THEN
+.....+
?.. Linelist @ I := link Backtrack ( ( I_fname pick Allpathssortedbyfname
) str_remove_substr D_webraw )
-->[nextv]
+-+--------++-+--------------------+-+-+-++
| ||"||
+-+--------++-+--------------------+-+-+-++
>> '../../' should NOT be in there - how did this happen?
I added :
liner := line str_remove_subStr '[#=; backtrack ;=#]' ;
After much floundering :
?.. Linelist @ I := link Backtrack ( ( I_fname pick Allpathssortedbyfname
) str_remove_substr D_webraw )
-->[nextv] i_fname
2477
-->[nextv]
+-+--------++-+-----------------------------------------------------------------------------------------------
| ||"||
---------+-+-+-++
OK, now subDirs are added
24************************24
16Nov2020 webPageSite_update, [str, path]_executeEmbeds
current issues :
'[#=; backtrack ;=#]' rather than n*'../'
bodyLinks lack subDir!
'strings.ndf' Change :
+.....+
strList#midIndxs := EACH execute midLinks ;
+.....+
To :
+.....+
midLinks := EACH execute midLinks ;
+.....+
qnial> flag_break := l
l
qnial> webPageSite_update_test o
subDirfname = Pandemics, health, and the Sun/influenza/Howell - influenza virus.html
>> the break in str_executeEmbeds never occurs, WHY?
midIndxs strList := str_splitLftRgtTo_midIndxs_StrList '[#=; ' ' ;=#]' strNew ;
-->[nextv] midIndxs strList
++----------------------------------------------------------------+
||+---------------------------05-----05---------05-----05------------+|
||| ||
||+---------------------------05-----05---------05-----05------------+|
++----------------------------------------------------------------+
>> why is midIndxs null?
Run str_splitLftRgtTo_midIndxs_StrList tests, something must have changed?
I have to create a test in link d_Qtest 'strings- tests.ndf'
Check str_splitWith_subStr_test
All 12 work EXCEPT the 2 with '[#=; '!!
qnial> str_splitLftRgtTo_midIndxs_StrList_test
#05-----05
str_splitLftRgtTo_midIndxs_StrList_test, Mon Nov 16 10:30:23 2020
# string_splitWith_string_test example 1 : FAILED - result does NOT match standard
05-----05-----+------------------------------------------------------------------+
|[#=; | ;=#]||
05-----05-----+------------------------------------------------------------------+
+---------05-----05---------05-----05--------------------------------------+
||
+---------05-----05---------05-----05--------------------------------------+
++------------------------------------------------------------------------+
||+---------05-----05---------05-----05--------------------------------------+|
|||||
||+---------05-----05---------05-----05--------------------------------------+|
++------------------------------------------------------------------------+
qnial> loaddefs link d_Qtest 'strings- tests.ndf'
>>> loading start : strings- tests.ndf
<<< loading ended : strings- tests.ndf
qnial> str_splitLftRgtTo_midIndxs_StrList_test
#05-----05
str_splitLftRgtTo_midIndxs_StrList_test, Mon Nov 16 10:31:04 2020
# string_splitWith_string_test example 1 : FAILED - result does NOT match standard
05-----05-----+------------------------------------------------------------------+
|[#=; | ;=#]||
05-----05-----+------------------------------------------------------------------+
+-+------------------------------------------------------------------------+
|2|+---------05-----05---------05-----05--------------------------------------+|
| ||||
| |+---------05-----05---------05-----05--------------------------------------+|
+-+------------------------------------------------------------------------+
++------------------------------------------------------------------------+
||+---------05-----05---------05-----05--------------------------------------+|
|||||
||+---------05-----05---------05-----05--------------------------------------+|
++------------------------------------------------------------------------+
>> Interesting, missing midIndxs. what happened?
str_splitWith_subStr_test : 8 tests are OK
break in str_splitLftRgtTo_midIndxs_StrList_test
str_splitLftRgtTo_midIndxs_StrList Change :
+.....+
IF (OR EACH OR (null EACHRIGHT = i_heads i_tails))
+.....+
To :
+.....+
IF (OR (null EACHRIGHT = i_heads i_tails))
+.....+
I think the change was to make weird heasdTails work
str_splitLftRgtTo_midIndxs_StrList IS OP strLft strRgt str
{ LOCAL i i_heads i_tails midIndxs splits valids ;
IF flag_break THEN BREAK ; ENDIF ;
splits := link (strRgt EACHRIGHT str_splitWith_subStr (str_splitWith_subStr strLft str) ) ;
midIndxs := tell (gage shape splits) ;
i_heads := (strLft EACHRIGHT = splits) sublist midIndxs ;
i_tails := (strRgt EACHRIGHT = splits) sublist midIndxs ;
IF (OR (null EACHRIGHT = i_heads i_tails))
THEN null (fault '?str_splitLftRgtTo_midIndxs_StrList error : OR[i_heads, i_tails] is null')
ELSE
valids := 3 + (((i_heads + 4) EACHLEFT in i_tails) sublist i_heads) ;
valids splits
ENDIF
}
str_splitLftRgtTo_midIndxs_StrList Change :
+.....+
valids := 3 + (((i_heads + 4) EACHLEFT in i_tails) sublist i_heads) ;
+.....+
To :
+.....+
valids := 1 + (((i_heads + 2) EACHLEFT in i_tails) sublist i_heads) ;
+.....+
>> OK, this works. But it must have "damaged" the other application.
Add more str_splitLftRgtTo_midIndxs_StrList_test & re-test
['> perhaps because midInxs is a list?!
>> OK, that was that problem.
Now to change str_splitLftRgtTo_midIndxs_StrList for ['> OK, now all 5 str_splitLftRgtTo_midIndxs_StrList_test are OK
Double-check other link d_Qtest 'strings- tests.ndf' :
I changed outputs of all tests to use :
test_comment t_name t_input t_standard t_result ;
>> I need to fix the format!
qnial> strings_alltest
link d_Qtest '201116 12h25m44s alltest strings.txt' :
+----+
Summary of test results :
/media/bill/PROJECTS/Qnial/code develop_test/201116 12h25m44s alltest strings.txt, date= 201116 12h25m
# str_to_unicodeList_test example #1 : OK - result matches standard
# str_to_unicodeList_test example #2 : OK - result matches standard
# str_to_unicodeList_test example #3 : OK - result matches standard
# str_to_unicodeList_test example #4 : OK - result matches standard
# str_to_unicodeList_test example #5 : OK - result matches standard
# str_to_unicodeList_test example #6 : OK - result matches standard
# string_sub_test example #1 : OK - result matches standard
# string_sub_test example #2 : OK - result matches standard
# string_sub_test example #3 : OK - result matches standard
# string_sub_test example #4 : OK - result matches standard
# string_sub_test example #5 : OK - result matches standard
# string_sub_test example #6 : OK - result matches standard
# string_sub_test example #7 : OK - result matches standard
# string_sub_test example #8 : OK - result matches standard
# string_sub_test example #9 : OK - result matches standard
# string_sub_test example #10 : OK - result matches standard
# string_sub_test example #11 : OK - result matches standard
# string_sub_test example #12 : OK - result matches standard
# string_sub_test example #13 : OK - result matches standard
# str_splitWith_subStr_test example #1 : OK - result matches standard
# str_splitWith_subStr_test example #2 : OK - result matches standard
# str_splitWith_subStr_test example #3 : OK - result matches standard
# str_splitWith_subStr_test example #4 : OK - result matches standard
# str_splitWith_subStr_test example #5 : OK - result matches standard
# str_splitWith_subStr_test example #6 : OK - result matches standard
# str_splitWith_subStr_test example #7 : OK - result matches standard
# str_splitWith_subStr_test example #8 : OK - result matches standard
# str_splitLftRgtTo_midIndxs_StrList_test example #1 : OK - result matches standard
# str_splitLftRgtTo_midIndxs_StrList_test example #2 : OK - result matches standard
# str_splitLftRgtTo_midIndxs_StrList_test example #3 : OK - result matches standard
# str_splitLftRgtTo_midIndxs_StrList_test example #4 : OK - result matches standard
# str_splitLftRgtTo_midIndxs_StrList_test example #5 : OK - result matches standard
+----+
24************************24
15Nov2020 [str, path]_executeEmbeds
06:20 [develop, test] new code with 'Howell - influenza virus.html'
Key upgrades to :
#] str_executeEmbeds IS OP str phraseValueList - execute embeds in line, return a str
#] path_executeEmbeds IS OP path phraseValueList - execute embeds in a file
'Howell - influenza virus.html'
New optrs :
#] executeEmbedsGet_pathTemp IS - generate a unique p_temp fname
#] path_executeEmbedsInsertIn_fHand IS OP path phraseValueList - execute embeds, insert in fileHandle
14:34 webPageSite_update_test with 'Howell - influenza virus.html'
02--02
qnial> webPageSite_update_test o
?type error in fault
?type error in fault
?type error in fault
?type error in fault
?type error in fault
p_webSite = /media/bill/HOWELL_BASE/Website/Pandemics, health, and the Sun/influenza/Howell - influenza virus.html
subDir = Pandemics, health, and the Sun/influenza/
02--02
>> NO [embeds, links] survived,
>> later : nyet -
the backtracks survived as original, no processing
the whole-line-embeds disappeared entirely
'Website updates.ndf' Change :
+.....+
line := str_executeEmbeds line fout backtrack ;
+.....+
To :
+.....+
line := str_executeEmbeds line (("fout fout)("backtrack backtrack)) ;
+.....+
>> No faults, but no embeds either... internal links no longer in webPage
Check within-line-embeds '[#=; '
break in str_executeEmbeds
02--02
qnial> webPageSite_update_test o
>> no break occurred, same error?
24************************24
14Nov2020 fix up tests - a bit confusing
21:27
'Website updates.ndf', Change :
+.....+
line := str_executeEmbeds line fout backtrack ;
+.....+
To :
+.....+
line := str_executeEmbeds line (("fout fout) ("backtrack backtrack)) ;
+.....+
> I probably need a list of paired (("phrase value)...)
16:57 str_executeEmbeds isrepeating the footer part multiple times?
looking at a typical full-line embed, which is not set up properly :
02--02
[#!: str_executeEmbeds (link d_webWork 'fin Head_one.html') stdTmp d_webRaw d_webSite ; path_insertIn_fHand stdTmp fout ;
02--02
Note the recursive use of embedding.
[#!: strList_writeTo_path (str_executeEmbeds (link d_webWork 'fin Head_one.html')) (p_temp := link d_temp 'executeEmbed temp.txt') ; path_insertIn_fHand p_temp fout ;
????????????
16:52 I fixed path_retrieve_subDirFname
$ find "$d_Qroot" -maxdepth 3 -name "*.ndf" | tr \\n \\0 | xargs -0 -IFILE grep -w -i --with-filename --line-number 'path_retrieve_subDirFname' "FILE"
/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf:30: path_retrieve_subDirFname IS OP path dirBase - returns (subDir fname) from a path
/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf:245:IF flag_debug THEN write 'loading path_retrieve_subDirFname' ; ENDIF ;
/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf:247:#] path_retrieve_subDirFname IS OP path dirBase - returns (subDir fname) from a path
/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf:252: path_retrieve_subDirFname IS OP path dirBase
/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf:256: THEN fault '?path_retrieve_subDirFname error, dirBase not in path'
/media/bill/PROJECTS/Qnial/MY_NDFS/file_ops.ndf:260: THEN fault '?path_retrieve_subDirFname error, fname'
14Nov2020 11:49
$ find "$d_Qroot" -maxdepth 3 -name "*.ndf" | tr \\n \\0 | xargs -0 -IFILE grep -w -i --with-filename --line-number 'd_webDone' "FILE"
05-----05
/media/bill/PROJECTS/Qnial/MY_NDFS/Website updates.ndf:181:# Note that menuHeadFoot are NOT [copied, converted] to d_webDone, as they are useless there.
/media/bill/PROJECTS/Qnial/MY_NDFS/Website updates.ndf:192: NONLOCAL d_htmlBackup d_webRaw d_webDone ;
/media/bill/PROJECTS/Qnial/MY_NDFS/Website updates.ndf:265:# Note that menuHeadFoot are NOT [copied, converted] to d_webDone, as they are useless there.
/media/bill/PROJECTS/Qnial/MY_NDFS/Website updates.ndf:276: NONLOCAL d_htmlBackup d_webRaw d_webDone ;
/media/bill/PROJECTS/Qnial/MY_NDFS/Website updates.ndf:358: d_htmlBackup := link d_webDone 'z_Archive/' timestamp_YYMMDD_HMS ' backups/' ;
/media/bill/PROJECTS/Qnial/code develop_test/Website updates- tests.ndf:20: - for testing only (depends on [pinn, d_webRaw, d_webDone])
/media/bill/PROJECTS/Qnial/code develop_test/Website updates- tests.ndf:22: depther inputs (eg n times '../') are arbitrary - for testing only (depends on [pinn, d_webRaw, d_webDone])
/media/bill/PROJECTS/Qnial/code develop_test/Website updates- tests.ndf:60:#] - for testing only (depends on [pinn, d_webRaw, d_webDone])
/media/bill/PROJECTS/Qnial/code develop_test/Website updates- tests.ndf:234:#] depther inputs (eg n times '../') are arbitrary - for testing only (depends on [pinn, d_webRaw, d_webDone])
/media/bill/PROJECTS/Qnial/code develop_test/Website updates- tests.ndf:400: NONLOCAL d_htmlBackup d_webRaw d_webDone ;
/media/bill/PROJECTS/Qnial/code develop_test/Website updates- tests.ndf:419: NONLOCAL d_htmlBackup d_webRaw d_webDone ;
05-----05
>> replace d_webDone with d_webSite
*****
13Nov2020 thorough check of [menuHeadFoot, body]links
several menus are not working - even more subMenus fail (maybe most?)
bodylinks are NOT working, the subDirs are missing!
I revamped webPageRaw_update so that no [test, backup] files appear in d_webRaw - they are in d_htmlBackup.
The subDirs are still missing - may not be a problem as lon this is done for the webSite?
BUT - the code indicates that the full path SHOULD appear!!???
02--02
lineList@i := link backtrack
((i_fname pick allPathsSortedByFname) str_remove_subStr d_webRaw) ;
...
lineList@i := link backtrack
((i_subDir pick allSubDirsSortedBySubdir) str_remove_subStr d_webRaw) ;
02--02
Run a test webPageDone_update to see what happens.
05-----05
Olde code
# 13Nov2020 no longer used
dw_base := d_webRaw ;
ds_base := d_webDone ;
******
11Nov2020 update to website, check menuHeadFoots
webSite_update
etc, etc
>> webSite_update seems to work.
From a quick check - most menu items seem to work. A few exceptions were noted :
02--02
top menu - all looks good
subMenu Neural Networks :
Neural Nets - no works
MindCode - OK
subMenu Projects :
MindCode neural network - OK
Randell Mills- hydrinos - no works
Puetz - The Greatest of cycles - no works
rest of subMenus look good
02--02
>> a thorough check is needed, including bodyLinks!
Do a more thorough check Friday!
******
11Nov2020 webPage_update_test continued
Idiot - I chopped the " and not the >, so now changed to :
THEN line := internalLinks_return_relativePath backtrack 'this Directory's listing.
Check d_website version :
No update? - oops, at present, this goes to 'test- ' versions.
Leave it for debugging
>> test versions don't have menuHeadFoots
Now, run ALL webPage_update_test : check log file, and output files
>> still a problem with ./ eg :
You can see these files via this Directory's listing.
>> but only for [1,2]?
backup '1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
by dating
then rename 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
to '1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
>> This reduces the [known, good] changes from future diff outputs, making it easier to focus on real problems.
05-----05
Why is ./ still a problem? :
I added './' to midIndxsLines_bads
Removed from internalLinks_return_relativePath :
% check if ./ ;
ELSEIF (= './' lineList@i) THEN
null ;
Re-run webPage_update_test for only :
'economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
>>
05-----05
menuHeadFoot disappearance :
str_executeEmbeds
02--02
-->[nextv] str fout backtrack
+-------------------------------------------------------------------------------------------------------------
|[#!: pinn_writeExecute_pout (link d_webWork 'fin Head_one.html') stdTmp d_webRaw d_webSite ; path_ins
+-------------------------------------------------------------------------------------------------------------
------------------------------+-+---------+
ertIn_fHand stdTmp fout ; |4|../../../|
------------------------------+-+---------+
-->[nextv]
?undefined identifier: PINN_WRITEEXECUTE_POUT <***> ( LINK D_WEBWORK
02--02
>> Arrrggghhh! I forgot to change the embeds!!
Create a new fileops.ndf optr :
str_replaceIn_pathList IS OP flag_backup d_backup strOld strNew f_Pattern pathList
str_replaceIn_pathList IS OP flag_backup d_backup strOld strNew f_Pattern pathList
I did a huge amount of fixing :
created str_replaceIn_pathList
converted od optr name :
str_replaceIn_pathList d_webRaw 'pinn_writeExecute_pout' 'str_executeEmbeds' htmlPathsSortedByFname
24************************24
10Nov2020 webPage_update_test - continue to change, debug
Run (single example for development) :
webPage_update_test
oops - blew up
02--02
qnial> webPage_update_test
increasing call stack to 200
increasing call stack to 300
...
Segmentation fault
02--02
>> probably a path error or infinite loop
[webPage, pout] ARE checked to stop progress, so maybe an infinite loop?
24************************24
10Nov2020
05-----05
Problem with subDir faults - internalLinks_return_relativePath_test tests #[5 6]
More efficient to create a new list allEndSubDirsSortedByFullSubdir
Key changes to internalLinks_return_relativePath_test
>> now all tests work nicely!
05-----05
test webPage_update
# Set of tests :
# htmlFname := 'economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
# htmlFname := 'page Howell - blog.html'
# htmlFname := 'Pandemics, health, and the Sun/corona virus/Howell - corona virus.html'
# htmlFname := 'Lies, Damned Lies, and Scientists/_Lies, damned lies, and scientists.html'
# htmlFname := 'Pandemics, health, and the Sun/influenza/Howell - influenza virus.html'
# webPage_update (d_webRaw link htmlFname)
'page Howell - blog.html'
 and (1926-2020).png)
>> IMG SRC problem - it goes to ">, but should only go to the next "
Change :
+.....+
IF ('
THEN line := internalLinks_return_relativePath '<IMG SRC=) ' line ;
+.....+
To :
+.....+
IF ('> one step, but NO backtracks & subDir not provided for ![]() > ALL internalLinks_return_relativePath_test are OK.
>> But now for ALL test-output file :
no menueHeadFoot
'
' [nextv] line
 and (1926-2020).png)
?.. Midindxs Linelist := str_splitlftrgtto_midindxs_strlist Strleft
Strright Liner
-->[nextv] lineList
?no_value
-->[nextv]
++-------------------------------------------------------------------------+
||?str_splitLftRgtTo_midIndxs_StrList error : OR[i_heads, i_tails] is null|
++-------------------------------------------------------------------------+
fix str_splitLftRgtTo_midIndxs_StrList to use " = strRgt
Change :
+.....+
valids := 1 + (((i_heads + 2) EACHLEFT in i_tails) sublist i_heads) ;
+.....+
To :
+.....+
valids := 1 + (((i_heads + 4) EACHLEFT in i_tails) sublist i_heads) ;
+.....+
Also - remove " from strRght
Change str_splitLftRgtTo_midIndxs_StrList :
+.....+
valids := 2 + (((i_heads + 4) EACHLEFT in i_tails) sublist i_heads) ;
+.....+
To :
+.....+
valids := 3 + (((i_heads + 4) EACHLEFT in i_tails) sublist i_heads) ;
+.....+
Remove " from strLeft strRght in 'Website updates- tests.ndf'
Re-run :
internalLinks_return_backupSubDirFnames_test :
02--02
# internalLinks_return_backupSubDirFnames_test example #4 : FAILED - result does NOT match standard
gnuplot Ive used the unofficial extension .plt to designate gnuplot scripts for each of the graphs. You can see these files via this Directorys listing.
gnuplot Ive used the unofficial extension .plt to designate gnuplot scripts for each of the graphs. You can see these files via this Directorys listing.
02--02
>> now the ./ doesn't work - oops, I had forgotten to add it to internalLinks_return_backupSubDirFnames
internalLinks_return_relativePath_test :
02--02
# internalLinks_return_relativePath_test example #9 : FAILED - result does NOT match standard
 WIDTH=90% NAME="1872-2020 SP500 index semi-log detrended"
02--02
>> most files are OK
>> how did this get the wrong file????
Oops - I mixed up the standard with another example
>> Now ALL tests are OK
If possible, I need to consolidate
internalLinks_return_backupSubDirFnames - This is a "shorthand" version for editing, '[#=; backtrack ;=#]'
used by webPage_convertBodyLinks
internalLinks_return_relativePath - This is the "full" version for updating the webSite
used by webPage_update
For :
internalLinks_return_relativePath_BodyLinks_test
t_input := '[#=; backtrack ;=#]' blah blah
OK - more changes to consolidate [internalLinks_return_backupSubDirFnames, internalLinks_return_relativePath]
Re-run :
internalLinks_return_relativePath_BodyLinks_test :
02--02
[3,5,6,8,9] failed, but [3,5,6,9] are due to incorrect standards, given my current "full path" approach
- I fixed the standard response
[3,5,6,9] - now the full '> All are OK now
internalLinks_return_relativePath_test :
>> [1-12] all are OK
05-----05
Olde code
# OUTMODED! now outputs go into sa d_webRaw subDir as the webPage
IF flag_debug THEN write 'loading webPage_update_test' ; ENDIF ;
#] webPage_update_test IS - Check for proper processing of embedded executables
# These tests cannot be run from d_Qtests : I must create "test" output files as standards in d_webSite
# uncomment to run one test (insert line below, remove extra line for next test definition below)
webPage_update_test IS OP webPage
{ LOCAL comments flog i_test p_inn p_log p_std ;
flag_screenOut := o ;
p_log := link d_Qtest 'webPage_update_test log.txt' ;
path_backupDated_delete p_log ;
flog := open p_log "a ;
flog EACHRIGHT writefile '********' (link 'webPage_update_test, ' timestamp_DDMMMYYYY_HMS) '' ;
close flog ;
i_test := 0 ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_test_output flag_screenOut i_test p_inn p_std comments ;
}
# webPage_update_test_bag
webPage_update_test IS
{ LOCAL comments flog i_test p_inn p_log p_std ;
flag_screenOut := o ;
p_log := link d_Qtest 'webPage_update_test log.txt' ;
path_backupDated_delete p_log ;
flog := open p_log "a ;
flog EACHRIGHT writefile '********' (link 'webPage_update_test, ' timestamp_DDMMMYYYY_HMS) '' ;
close flog ;
i_test := 0 ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- page Howell - blog.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- page Howell - blog.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_test_output flag_screenOut i_test p_inn p_std comments ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_test_output flag_screenOut i_test p_inn p_std comments ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- Howell - corona virus.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- Howell - corona virus.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_test_output flag_screenOut i_test p_inn p_std comments ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- _Lies, damned lies, and scientists.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- _Lies, damned lies, and scientists.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_test_output flag_screenOut i_test p_inn p_std comments ;
webPage_update_output i_test p_inn p_std comments ;
}
# olde version
IF flag_debug THEN write 'loading internalLinks_return_backupSubDirFnames' ; ENDIF ;
#] internalLinks_return_backupSubDirFnames IS OP strLeft strRight line - returns backupFnameSubDir
#] for an an internal link, where SubDir goes into d_webSite
# Fixing links is noble, but perhaps equally important is [de-list, label]ing those that are flawed?
# and labelling "bad links" for manfixes later ;
# subDirs may be vulnerable to duplicates in different directories? Only the first match is used?
# EACH non-http link should have EITHER '[#=; backtrack ;=#]' or '!!linkError!!', but not both.
# I need to do more for paths with `#, as some van be "rescued". later ...
internalLinks_return_backupSubDirFnames IS OP strLeft strRight line
{ LOCAL i lineList fixIndxs_in fname midIndx midIndxs midLinks n_midLinks path subDir ;
NONLOCAL allPathsSortedByPath allFnamesSortedByFname ;
% ;
midIndxs lineList := str_splitLftRgtTo_midIndxs_StrList strLeft strRight line ;
midLinks := lineList#midIndxs ;
n_midLinks := gage shape midLinks ;
fixIndxs_in := n_midLinks reshape l ;
% most lines do not have [ gnuplot Ive used the unofficial extension .plt to designate gnuplot scripts for each of the graphs. You can see these files via this Directorys listing.'
# str_remove_subStr ' gnuplot Ive used the unofficial extension .plt to designate gnuplot scripts for each of the graphs. You can see these files via this Directorys listing.' 'hello'
# (solitary 'Past & future worlds.html') EACHLEFT find_howell Allfnamessortedbyfname
# loaddefs link d_Qndfs 'Website updates.ndf'
# olde code removed
(path_extract_dir pinn)
(path_extract_fname pinn)
07Nov2020 :
pouter := pout ;
IF (~= stdTmp pouter) THEN pouter := link d_out subDir ; ENDIF ;
depther_global := depther ;
continue := l ;
IF continue THEN
ENDIF ;
24************************24
09Nov2020 So, backtracks seem OK, but fname only, no paths!!!
re-run internalLinks_return_relativePath_test :
>> They ALL failed!!! (arghhh!)
quick tes
qnial> 1028 pick allPathsSortedByFname
/media/bill/SWAPPER/Website - raw/Software programming & code/bin/dir_size sum, net test.txt
>> OK
run test on one file
>> internalLinks_return_relativePath flag_break is not encountered for correct fname, no subDir
>> I had problem before!! reverted to fail
qnial> internalLinks_return_relativePath_test
#05-----05
internalLinks_return_relativePath_test, Mon Nov 9 17:08:26 2020
# internalLinks_return_relativePath_test example #1 : FAILED - result does NOT match standard
t_input, t_standard, t_result =
+---------02--02--------------------------------------------------------------+
|||
+---------02--02--------------------------------------------------------------+
?op_parameter
Oops!! - left-over for some reason?
Change :
+.....+
t_result := internalLinks_return_relativePath l t_input ;
+.....+
To :
+.....+
t_result := internalLinks_return_relativePath t_input ;
+.....+
Are '../../../' being removed?
No - I forgot to add that coding.
I applied remove to entire line of internalLinks_return_relativePath :
line := line str_remove_subStr '../' ;
midIndxs lineList := str_splitLftRgtTo_midIndxs_StrList strLeft strRight line ;
changed lineliner so it can be modified
Re-try internalLinks_return_relativePath_test :
>> Nuts - I got the midIndx wrong - it does refer to path
pathsIndxs := midIndxs + 3 ;
paths := lineList#pathsIndxs ;
pathList := null ;
I significantly simplified internalLinks_return_relativePath
I am missing a test for bads, eg from internalLinks_return_backupSubDirFnames :
IF (OR (midIndxsLines_bads EACHLEFT subStr_in_str path)) THEN
fixIndxs_in@i := o ;
ENDIF ;
I adapted this]
Re-try internalLinks_return_relativePath_test :
24************************24
09Nov2020 webPage_update_test
After MANY, many changes, run test :
>> OOPS :
1. 'A HREF="' etc removed
2. only '../' appears
3. p_log doesn't work
1. internalLinks_return_relativePath - removed :
% remove [#= =#] brackets ;
shaper := gage shape midIndxs ;
lineList#(midIndxs - 1) := shaper reshape (solitary null) ;
lineList#(midIndxs + 1) := shaper reshape (solitary null) ;
2. in webPage_update :
02--02
depther_global := 0 ;
IF (OR ('Menu' 'fin Head' 'fin Footer' 'fin footer' EACHLEFT subStr_in_str pinn))
THEN depther := depther_global ;
ELSE depther := (gage shape (`/ findAll_Howell pinn ))
- (gage shape (`/ findAll_Howell d_webRaw));
ENDIF ;
02--02
>> but d_Qtests is used, NOT d_webRaw
U could put in something like :
02--02
ELSEIF (d_Qtest subStr_in_str pinn) THEN
depther := (gage shape (`/ findAll_Howell pinn ))
- (gage shape (`/ findAll_Howell d_webRaw));
02--02
>> but this is just complicates everything.
Just leave as is and rely on actual webPage updates to check backtracks
3. flag_test is currently set in webPage_update_test_output, so it's OK :
webPage_update l p_inn p_temp_webPageUpdate ;
Re-run webPage_update_test_output with log output and flag_tst := o
This allows diff to compare.
Interesting - stray `> gave double-HREF!?!?
In addition to the video of the presentation, "Howell 161220 Big Data, Deep Learning, Safety.ogv">, I have also
In addition to the video of the presentation, "Howell 161220 Big Data, Deep Learning, Safety.ogv!!linkError!!">, I have also posted
>> worry about this later - good that '!!linkError!!' flags it.
05-----05
Try all tests
qnial> webPage_update_test
diff: /media/bill/PROJECTS/Qnial/code develop_test/test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html update.html: No such file or directory
was missing 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html update.html'
>> copied from p_temp_webPageUpdate
Redo tests, but with flag_screenOut
# htmlFname := 'economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
# webPage_update l (d_webRaw link htmlFname) (d_webSite link htmlFname)
>> seems OK
>> retried after failures below - still seems to work
# htmlFname := 'page Howell - blog.html'
# webPage_update l (d_webRaw link htmlFname) (d_webSite link htmlFname)
>> total failure of backtracks
>> wait - root wbPages won't have backtracks, but then we sould see a full oath!!
# htmlFname := 'Pandemics, health, and the Sun/corona virus/Howell - corona virus.html'
# webPage_update l (d_webRaw link htmlFname) (d_webSite link htmlFname)
>> total failure of backtracks
>> [#=; backtrack ;=#] not in webPage? (did I run webPage_convertBodyLinks?)
>> wait - root wbPages won't have backtracks, but then we sould see a full oath!!
# htmlFname := 'Lies, Damned Lies, and Scientists/_Lies, damned lies, and scientists.html'
# webPage_update l (d_webRaw link htmlFname) (d_webSite link htmlFname)
>> backtrack OK, but fname only, no paths!!!
# htmlFname := 'Pandemics, health, and the Sun/influenza/Howell - influenza virus.html'
# webPage_update l (d_webRaw link htmlFname) (d_webSite link htmlFname)
>> backtrack OK, but fname only, no paths!!!
05-----05
So, backtracks seem OK, but fname only, no paths!!!
stop now - supper and preps for FireBoD meeting
24************************24
09Nov2020
05-----05
Problem with backtrack (was due to faulty code, see below) :
# tests - playing with backtrack
# backtrack := link chr_apo '../../../' chr_apo ' ' chr_apo 'banana' chr_apo
# backtrack := link 'pass ' chr_apo '../../../' chr_apo
# backtrack := link chr_apo '../../../' chr_apo
# execute backtrack
# olde code
%strList#midIndxs := (chr_apo EACHRIGHT link strList#midIndxs) EACHLEFT link chr_apo ;
# tests - t_input
t_input := ''
1 (link 'pass ' chr_apo '../../../' chr_apo) ;
?.. Strlist # Midindxs := EACH execute Strlist # Midindxs
-->[nextv] backtrack
pass '../../../'
-->[nextv] execute backtrack
../../../
-->[nextv] each execute midLinks
+----------------+
|pass '../../../'|
+----------------+
-->[nextv] execute link 'pass ' chr_apo '../../../' chr_apo
../../../
>> why isn't this working?
-->[nextv] backtrack
pass '../../../'
-->[nextv] execute backtrack
../../../
#
02--02
t_input := ''
1 (link chr_apo '../../../' chr_apo) ;
>> close, but no cigar
02--02
t_input := ''
1 '5' ;
>> works fine, ts just a string that should remain a string (without quotes) that is a problem
>> after Change :
+.....+
strList#midIndxs := EACH execute strList#midIndxs ;
+.....+
To :
+.....+
strList#midIndxs := EACH execute midLinks ;
+.....+
?.. Strlist # Midindxs := EACH execute Midlinks
-->[nextv]
+--------------------------05-----05-----------05-----05---------------------------+
||
+--------------------------05-----05-----------05-----05---------------------------+
02--02
Now try, post-correction :
t_input := ''
1 '../../../' ;
# str_executeEmbeds_test example #1 : OK - result matches standard
t_input, t_standard, t_result =
+------------------------------------------------------------------------+-+---------+
||1|../../../|
+------------------------------------------------------------------------+-+---------+
05-----05
Remove :
# loaddefs link d_Qtest 'Website updates- tests.ndf'
IF flag_debug THEN write 'loading webPage_update_output' ; ENDIF ;
# webPage_update_output IS - write results to log file
# 07Nov2020 initial
webPage_update_output IS OP i_test p_inn p_std comments
{ LOCAL flog p_temp_BodyLinks ;
NONLOCAL d_webRaw d_webSite ;
p_temp_webPageUpdate := link d_temp 'webPageUpdate temp.txt' ;
p_log := link d_Qtest 'webPage_update_test log.txt' ;
flog := open p_log "a ;
flog EACHRIGHT writefile
'........' (link '# webPage_update example #' (string i_test))
(link 'webPage_update_test for : "' (path_extract_fname p_inn) '"') ;
flog EACHRIGHT writefile comments ;
writefile flog 'diff results : ' ;
close flog ;
% pinn_executeEmbeddedTo_pout has the [dir, file] existence checks, so no need here ;
pinn_executeEmbeddedTo_pout p_inn p_temp_webPageUpdate d_webRaw d_webSite ;
host (link 'diff --width=85 "' p_std '" "' p_temp_webPageUpdate '" --suppress-common-lines | grep ' chr_apo '^>' chr_apo ' | sed ' chr_apo 's/^>\ //' chr_apo ' >>"' p_log '"') ;
host link 'echo "" >>"' p_log '"' ;
% 07Nov2020 manually move p_temp_webPageUpdate to p_std as required ;
}
# loaddefs link d_Qtest 'Website updates- tests.ndf'
IF flag_debug THEN write 'loading webPage_update_test' ; ENDIF ;
# webPage_update_test IS - change links in the body of html to the relative format
# This doesn't help at all, as I can't test the links unless a file is in d_webSte!!
# 07Nov2020 test created
webPage_update_test IS
{ LOCAL comments flog i_test p_inn p_log p_std ;
p_log := link d_Qtest 'webPage_update_test log.txt' ;
path_backupDated_delete p_log ;
flog := open p_log "a ;
flog EACHRIGHT writefile
'********' (link 'webPage_update_test, ' timestamp_DDMMMYYYY_HMS) '' ;
close flog ;
i_test := 0 ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- page Howell - blog.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- page Howell - blog.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_output i_test p_inn p_std comments ;
}
# webPage_update_test_bag
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- page Howell - blog.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- page Howell - blog.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_output i_test p_inn p_std comments ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_output i_test p_inn p_std comments ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- Howell - corona virus.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- Howell - corona virus.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_output i_test p_inn p_std comments ;
% ;
i_test := i_test + 1 ;
p_inn := link d_Qtest 'test- _Lies, damned lies, and scientists.html convertBodyLinks.html' ;
p_std := link d_Qtest 'test- _Lies, damned lies, and scientists.html update.html' ;
comments :=
'This file has many "", no "mailto:". Straightforward.'
'29Oct2020 - I still have to do a thorough check of the output, all cases.'
;
webPage_update_output i_test p_inn p_std comments ;
24************************24
08Nov2020 test & fix pinn_writeExecute_pout
05-----05
# Header example :
[#!: pinn_writeExecute_pout (link d_webWork 'fin Head_one.html') stdTmp d_webRaw d_webSite ; path_insertIn_fHand stdTmp fout ;
Howell web - Past and Future Worlds
[#!: pinn_writeExecute_pout (link d_webWork 'fin Head_two.html') stdTmp d_webRaw d_webSite ; path_insertIn_fHand stdTmp fout ;
[#!: pinn_writeExecute_pout (link d_webWork 'Menu.html') stdTmp d_webRaw d_webSite ; path_insertIn_fHand stdTmp fout ;
[#!: pinn_writeExecute_pout (link d_webWork 'Menu Howell videos.html') stdTmp d_webRaw d_webSite ; path_insertIn_fHand stdTmp fout ;
# For all webPages, should change to :
[#!: menuHeadFoot_writeTo_fout (link d_webWork 'fin Head_one.html') fout backtrack ;
Howell web - Past and Future Worlds
[#!: menuHeadFoot_writeTo_fout (link d_webWork 'fin Head_two.html') fout backtrack ;
[#!: menuHeadFoot_writeTo_fout (link d_webWork 'Menu.html') fout backtrack ;
[#!: menuHeadFoot_writeTo_fout (link d_webWork 'Menu Howell videos.html') fout backtrack ;
24************************24
07Nov2020
Abandoned as useless : created :
webPage_update_output IS - write results to log file
pinn_writeExecute_pout - I commented out :
% 0Nov2020 comment out : IF (~= stdTmp pouter) THEN pouter := link d_out subDir ; ENDIF ;
pouter
>> pouter is last line of optr
webPage_update_test on 'test- page Howell - blog.html convertBodyLinks.html' :
>> WRONG!!!, needs FULL subDir!!! most links are the same...
what is wrong?
nyet : I think its because pinn_writeExecute_pout is baffled by having dwebRaw & d_webSite in same dir?
nyet - useless : I created webPage_update_output to handle test webPage updates.
I removed the 'pouter' return of pinn_executeEmbeddedTo_pout!! (idiot)
02--02
Set of tests : see link d_Qndfs 'file_ops.ndf'
>> the full path of a file is NOT provided, nor are the '../'
Where is "backtrack"? It's in pinn_executeEmbedsTo_pout
I have to find the full path to [fnames, subDirs] in pinn_executeEmbedsTo_pout
Major adaptation from internalLinks_return_backupSubDirFnames
17:38 OK - loaddefs works, but probably not pinn_executeEmbedsTo_pout
Break for the day, do dishes.
24************************24
07Nov2020
05-----05
See how many linkErrors remain :
$ find "$d_webRaw" -maxdepth 3 -type f -name "*.html" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number '!!linkError!!' "FILE" | grep --invert-match "z_Archive"
02--02
These ones are known problems (probably never did create the files?) :
/media/bill/SWAPPER/Website - raw/page Howell - blog.html:75:
02--02
These ones shouldn't be a problem, but... :
fname problems? :
/media/bill/SWAPPER/Website - raw/page Howell - blog.html:763:
>> just recopy path
/media/bill/SWAPPER/Website - raw/page Howell - blog.html:888:
>> oops, apo
/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/Howell - pandemics and disease blog.html:45:
profile
challenged, and when they crumble
and Scientific Thinking?
B2 - Cheating theory and Game theory
B3 - Pre-and-post-Science Philosophies for Thinking
C1 - A Brave new world
C2 - The rise & fall of Enlightenment
C3 - Suggestions for science, policy, and society
D0 - Conclusions
24************************24
05Nov2020 webSite_convertBodyLinks
Code modified to [capture process p_log, backup files, log diffs]
02--02
% create a new backup directory for every use of webSite_convert, as damage can be VERY time-costly ;
d_htmlBackup := link d_webRaw 'z_Archive/' timestamp_YYMMDD_HMS ' backups/' ;
host link 'mkdir "' d_htmlBackup '" ' ;
% ;
flog := open p_log "w ;
flog EACHRIGHT writefile
'********' (link 'webSite_convertBodyLinks, ' timestamp_DDMMMYYYY_HMS) '' ;
close flog ;
FOR webPage WITH (strList_readFrom_path p_webPageList) DO
fname := path_extract_fname webPage ;
p_backup := link d_htmlBackup fname ;
flog := open p_log "a ;
flog EACHRIGHT writefile '........' (link 'webPage : "' fname '"') 'diff results : ' ;
close flog ;
% webPage_convertBodyLinks o webPage d_webRaw ;
% host (link 'diff --width=85 "' p_backup '" "' webPage '" --suppress-common-lines | grep ' chr_apo '^>' chr_apo ' | sed ' chr_apo 's/^>\ //' chr_apo ' >>"' p_log '"') ;
host link 'echo "" >>"' p_log '"' ;
ENDFOR ;
02--02
05-----05
qnial> webSite_convertBodyLinks
2_website htmlFileLs.txt :
/media/bill/SWAPPER/Website - raw/Projects - mini/Diversity - ssh site/diversity_public/home.html
/media/bill/SWAPPER/Website - raw/Projects - mini/Solar system/Cdn Solar Forecasting/Canadian Solar Workshop 2006 home page.html
/media/bill/SWAPPER/Website - raw/Projects - mini/Solar system/Cdn Solar Forecasting/CSWProgram.html
/media/bill/SWAPPER/Website - raw/Projects - mini/wordpress site/Authors Guide BLOG home.html
/media/bill/SWAPPER/Website - raw/security/encryption-decryption instructions.html
/media/bill/SWAPPER/Website - raw/Software programming & code/Qnial/MY_NDFS/email Thunderbird - Base64 Encode and Decode Base64 Files, instructions.html
/media/bill/SWAPPER/Website - raw/Software programming & code/Qnial/MY_NDFS/Thunderbird - Base64 Encode and Decode Base64 Files, instructions.html
/media/bill/SWAPPER/Website - raw/Software programming & code/Qnial/QNial.html
/media/bill/SWAPPER/Website - raw/Solar modeling and forecasting/_Solar modeling & forecasting.html
/media/bill/SWAPPER/Website - raw/Steven H Yaskell/0_Steven H Yaskell.html
/media/bill/SWAPPER/Website - raw/webWork files/4_test Kyoto Premise - the scientists arent wearing any clothes (copy).html
/media/bill/SWAPPER/Website - raw/webWork files/fin organisations.html
201105 17h55m50s webSite_convertBodyLinks log.txt :
webPage : "home.html"
webPage : "Canadian Solar Workshop 2006 home page.html"
webPage : "CSWProgram.html"
webPage : "Authors Guide BLOG home.html"
webPage : "encryption-decryption instructions.html"
webPage : "email Thunderbird - Base64 Encode and Decode Base64 Files, instructions.html"
webPage : "Thunderbird - Base64 Encode and Decode Base64 Files, instructions.html"
webPage : "QNial.html"
webPage : "_Solar modeling & forecasting.html"
webPage : "0_Steven H Yaskell.html"
webPage : "4_test Kyoto Premise - the scientists arent wearing any clothes (copy).html"
Only one NOT in 'webSite_convertBodyLinks log.txt : ' :
/media/bill/SWAPPER/Website - raw/webWork files/fin organisations.html
>> worry about later
05-----05
Now to uncomment :
% webPage_convertBodyLinks o webPage d_webRaw ;
% host (link 'diff --width=85 "' p_backup '" "' webPage '" --suppress-common-lines | grep ' chr_apo '^>' chr_apo ' | sed ' chr_apo 's/^>\ //' chr_apo ' >>"' p_log '"') ;
qnial> webSite_convertBodyLinks
>> OOPS! backups are NOT being made!!
>> The actual backups are done by webPage_convertBodyLinks, whick I hadn't activated
Try with flag_backup, but not yet flag_move
qnial> webSite_convertBodyLinks
>> NO diffs? Maybe the last time I over-wrote the webPages, it was done properly?
05-----05
Big mistake with my find in this section : ignore it all
$ find "$d_webRaw" -maxdepth 3 -type f -name "*.html" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number '!!linkError!!' "FILE" >"$d_webRaw""webWork files/5 linkerrors.xt"
>> using "!!linkError!!" gave jillions of :
grep: invalid max count
>> OCH!! tons of '!!linkError!!'
This result is wrong - I must have a coding error that generates this problem?
Many are the result of a lack of `/ at the end of a subDir
Maybe due to eliminating d_webaw as "base dir"!!!???!!!
467 cases in total - I have to sed these out and re-translate!!
find ONLY in recent backup dir
$ find "$d_webRaw""z_Archive/201105 18h37m51s backups/" -maxdepth 0 -type f -name "*.html" | tr \\n \\0 | xargs -0 -IFILE grep --with-filename --line-number '!!linkError!!' "FILE" >"$d_webRaw""webWork files/5 linkerrors.xt"
Nyet - reinstate "$d_webRaw""z_Archive/201105 18h37m51s backups/"
fix problem with webPage_convertBodyLinks
break for the day!
24************************24
02Nov2020 test "real" webPages to make sure they are saved, and that other problems don't arise.
Adjust link d_Qtest 'Website updates.ndf' and cold loaddef :
Uncomment in webPage_convertBodyLinks :
% host link 'mv "' p_temp_webPage_convertEncoding '" "' webPage '"' ;
Set flag_backup := l
qnial> bye / qnial
qnial> loaddefs link d_Qtest 'Website updates.ndf'
define pinn - use one of already-tested webPages, example :
qnial> pinn := link d_webRaw 'page Howell - blog.html'
qnial> webPage_convertBodyLinks l pinn d_webRaw
analyse the results
- check diff output
- open both the [raw, bodyLinked] versions, visually compare by searching 'A >"' p_log '"'))
) EACHLEFT link chr_apo ;
write writefileList ;
EACH host writefileList ;
05-----05
qnial> pinn := link d_webRaw 'page Howell - blog.html'
qnial> webPage_convertBodyLinks l pinn d_webRaw
# $ diff --width=85 "$d_webRaw""page Howell - blog.html" "$d_Q_tests""test- page Howell - blog.html convertBodyLinks.html" --suppress-common-lines
Issues :
+-+
+-+
05-----05
qnial> pinn := link d_webRaw 'economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html'
qnial> webPage_convertBodyLinks l pinn d_webRaw
# $ diff --width=85 "$d_webRaw""economics, markets/SP500/multi-fractal/1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html" "$d_Q_tests""test- 1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html convertBodyLinks.html" --suppress-common-lines
Issues :
+-+
+-+
05-----05
qnial> pinn := link d_webRaw 'Pandemics, health, and the Sun/corona virus/Howell - corona virus.html'
qnial> webPage_convertBodyLinks l pinn d_webRaw
# $ diff --width=85 "$d_webRaw""Pandemics, health, and the Sun/corona virus/Howell - corona virus.html" "$d_Q_tests""webPage_convertEncoding temp.txt" --suppress-common-lines
Issues :
+-+
+-+
24************************24
02Nov2020 link_fixErrs - remove ../ and fix many links
loaddefs link d_Qtest 'Website updates- tests.ndf'
internalLinks_return_backupSubDirFnames_test
>> OK, seems to work fine now on all tests
webPage_convertBodyLinks_test
>> I no longer have diff results? lost the code
I threw quick code up - doesn't see, to be working?...
05-----05
First test :
p_inn := link d_Qtest 'test- Howell - corona virus.html' ;
>> looks good, with notable issues :
02--02
>> Most (10) bodyLinks work well, eg :
89,90c87,88
< Anglophone |
< Scandanavia |
---
> Anglophone
> Scandanavia |
02--02
>> Some IMG work, eg :
256c254
< 
---
> 
272c270
< 
---
> 
02--02
>> Why didn't these work? IMG may not be working? Maybe the files were renamed?
>> Yikes! It appears that I overwrote the test file father than the pocessed file.
>> Many already have !!linkError!! although the program should handle these.
179c177
< 
---
> 
181c179
< 
---
> 
02--02
>> Two cases where subDirs are included when that is not necessary?
/media/bill/SWAPPER/Website - raw/Software programming & code/Qnial/code develop_test/test- Howell - corona virus.html str_replaceIn_path.html:177:
>> ALL of these should work!
>> NO, NOT for webPageSite_update!!?? It only inserets backtrack?
02--02
Howell 920000 Front - Introduction.pdf
qnial> (= 'Howell 920000 Front - Introduction.pdf' allFnamesSortedByFname ) sublist allPathsSortedByFname
>> again - no hits? It IS in :
'/media/bill/HOWELL_BASE/Website/Pandemics, health, and the Sun/corona virus'
try links via geany of 'webSite urlList.txt' :
Howell 920000 Front - Introduction.pdf
/media/bill/HOWELL_BASE/Website/Lies, Damned Lies, and Scientists/Howell 920000 Front - Introduction.pdf
>> This is in that directory, so why the failure?
02--02
NUTS! I haven't updated using webSite_extract_pathsSubDirsFnames?
>> that shouldn't be the problem!
I guess the good news is that many of the intern fails have 2-3 fails per path.
So it may not be as badit looks
Why isn't internalLinks_return_relativePath working now?
My changes yesterday must have screwed it up. Hard to track though, even with notes.
From 20Nov2020 :
internalLinks_return_relativePath :
Change :
+.....+
ELSEIF (NOT isfault
(i_fname := find_Howell
(fname := (1 + (last findAll_Howell `/ lineList@midIndx)) drop lineList@midIndx)
allFnamesSortedByFname
)
) THEN
lineList@midIndx := link backtrack
((i_fname pick allPathsSortedByFname) str_remove_subStr d_webRaw) ;
+.....+
>> This looks suspicious?
To :
+.....+
ELSEIF (NOT isfault
(i_fname := find_Howell
(1 + (last findAll_Howell `/ lineList@midIndx))
allFnamesSortedByFname
)
) THEN
lineList@midIndx := link backtrack
((i_fname pick allPathsSortedByFname) str_remove_subStr d_webRaw) ;
+.....+
>> Why did I drop lineList@midIndx? Might be a leftover?
Re-[loaddefs, internalLinks_return_relativePath_test] :
>> Now #2 also fails (!!linkError!!)
>> problem rains - it seems that the step above is being ignored
To :
+.....+
ELSEIF (NOT isfault
(i_fname := find_Howell
((1 + (last findAll_Howell `/ lineList@midIndx)) drop lineList@midIndx)
allFnamesSortedByFname
)
) THEN
lineList@midIndx := link backtrack
((i_fname pick allPathsSortedByFname) str_remove_subStr d_webRaw) ;
+.....+
Now Change (add "first") To :
+.....+
ELSEIF (NOT isfault
(i_fname := first find_Howell
((1 + (last findAll_Howell `/ lineList@midIndx)) drop lineList@midIndx)
allFnamesSortedByFname
)
) THEN
lineList@midIndx := link backtrack
((i_fname pick allPathsSortedByFname) str_remove_subStr d_webRaw) ;
+.....+
08********08
23Nov2020 Check [webPageSite, webPageOnln] etc for errors
First - fix some problems, but not ones that require webAllRawOrSite_update, which I will do after
>> OK for n0w - I only fixed one or two below
qnial> webSite_doAll
>> OUCH!! backtrack problem again - often the wrong number of'../'
>> Conference guides are included!??! - possibly because I changed --max-depth to 4
I have now REMOVED the max-depth!!
the invert-match should exclude ["z_Old, z_Archive, System_maintenance] anyways, but I removed Conference Guides exclusion. I might asl have it in as well - changes for next round...
--invert-match "Conference guides\|z_Old\|z_Archive\|System_maintenance\|Qnial_bag\|Cool emails/\|Electric Universe/References/\|Electric Universe/References/\|Yoonsuck Choe - conf program book/\|fin Head\|Menu\|fin [F,f]ooter\|fin organisations\|i9018xtp.default/extensions/\|[0-9]\{6\} [0-9]\{2\}h[0-9]\{2\}m[0-9]\{2\}s "
Big issue is the failed backtracks...
Re-try :
qnial> webSite_doAll
Wait a minutewebPageSites were updated, but not webPageRawes. Is this normal?
>> NO - webPageRawe_update is supposed to process all webPageRawes!!
Why didn't it?
webPageRawe_update IS OP flag_backup webPage
IF flag_backup THEN
host link 'mv "' p_temp '" "' webPage '"' ;
ENDIF ;
webSite_doAll IS
{
str_replaceIn_pathList l d_webRawe '!!linkError!!' '[#=; backtrack ;=#]' htmlPathsSortedByPath
webAllRawOrSite_update l "webPageRawe_update ;
webAllRawOrSite_update l "webPageSite_update ;
webURLs_extract ;
EACH urls_check 'intern' 'extern' ;
webSite_link_counts ;
}
>so this should be changing every webPageRawe!
last updates Sat 21Nov2020
Not this? :
subDir fname := path_retrieve_subDirFname webPage d_webRawe ;
fileops.ndf :
path_retrieve_subDirFname IS OP path dirBase
{ LOCAL fname fPath subDir ;
NONLOCAL webSiteAllPathList ;
IF (NOT (dirBase subStr_in_str path))
THEN fault '?path_retrieve_subDirFname error, dirBase not in path'
ELSE
fname := path_extract_fname path ;
IF (isfault fname)
THEN fault '?path_retrieve_subDirFname error, fname'
ELSE subDir fname := ((path str_extractPast_strFront dirBase) str_remove_subStr fname) fname
ENDIF
ENDIF
}
OUCH!! changed around arguments? :
str_replaceIn_path IS OP flag_backup d_backup strOld strNew path
str_replaceIn_pathList IS OP flag_backup d_backupRoot strOld strNew pathList
ELSE str_replaceIn_path o '' strOld strNew pinn ;
>> No, this is OK
webPageRawe_update Change :
+.....+
IF flag_backup
THEN IF (`m in d_htmlBackup)
THEN webPage path_backupTo_dir d_htmlBackup 'Rawe' ;
ELSE webPage path_backupDatedTo_dir d_htmlBackup ;
ENDIF ;
ENDIF ;
+.....+
To :
+.....+
IF flag_backup
THEN IF (`m in d_htmlBackup)
THEN webPage path_backupTo_dir d_htmlBackup ;
ELSE webPage path_backupDatedTo_dir d_htmlBackup ;
ENDIF ;
ENDIF ;
+.....+
webPageRawe_update Change :
+.....+
webPageRawe_update IS OP flag_backup webPage
...
IF flag_backup THEN
host link 'mv "' p_temp '" "' webPage '"' ;
ENDIF ;
+.....+
To :
+.....+
webPageRawe_update IS OP flag_backup flag_update webPage
...
IF flag_update THEN host link 'mv "' p_temp '" "' webPage '"' ; ENDIF ;
+.....+
I will also thatve to change
str_replaceIn_path IS OP flag_backup d_backup strOld strNew path
{ LOCAL finn fout flag_chg line p_temp ;
% ;
% backup unless a test, or called by [str_replaceIn_dir, str_replaceIn_pathList] which already backup ;
IF flag_backup THEN path path_backupDatedTo_dir d_backup ; ENDIF ;
p_temp := link d_temp 'str_replaceIn_path temp.txt' ;
flag_chg := o ;
% ;
finn := open path "r ;
fout := open p_temp "w ;
WHILE (NOT isfault (line := readfile finn)) DO
IF (strOld subStr_in_str line)
THEN
IF (= o flag_chg)
THEN flag_chg := l ;
%write path ;
ENDIF ;
EACH write '' line ;
line := str_replace_subStr strOld strNew line ;
write line ;
ENDIF ;
writefile fout line ;
ENDWHILE ;
EACH close finn fout ;
% ;
IF flag_backup THEN host link 'mv "' p_temp '" "' path '"' ; ENDIF ;
}
str_replaceIn_pathList IS OP flag_backup d_backupRoot strOld strNew pathList
{ LOCAL d_backup pinn ;
% backups are automatically done, except for testing purposes!! ;
IF flag_backup
THEN d_backup := link d_backupRoot 'z_Archive/' timestamp_YYMMDD_HMS
' backups str_replaceIn_pathList/' ;
host link 'mkdir "' d_backup '" ' ;
ENDIF ;
% ;
IF (NOT path_exists '-d' d_backup)
THEN EACH write '?str_replaceIn_pathList error : could not create d_backup : ' d_backup ;
ENDIF ;
FOR pinn WITH pathList DO
IF (NOT path_exists ("r pinn))
THEN EACH write
''
'?str_replaceIn_pathList error, file unknown : '
pinn
'' ;
% keep flag_backup = o (false) because all targeted files are backed up above ;
ELSE str_replaceIn_path flag_backup d_backup strOld strNew pinn ;
ENDIF ;
ENDFOR ;
}
5-----5
webPageSite :
file:///media/bill/HOWELL_BASE/Website/index.html
no images..
file:///media/bill/HOWELL_BASE/Website/page%20Publications%20&%20reports.html
no menu - execute embed only
file:///media/bill/HOWELL_BASE/Website/Neil%20Howell/_Neil%20Howell.html
no images, menu, etc
Projects subMenus
'S&P500 1872-2020, 83y trend' shows 'file:///media/bill/HOWELL_BASE/Website/economics,%20markets/SP500/multi-fractal/1872-2020%20SP500%20index,%20ratio%20of%20opening%20price%20to%20semi-log%20detrended%20price.html'
>> wrong link!
'Pandemics, health, Sun : '
'Fun, crazy stuff' -> change this 'Astro-correlates of health'
'Anthony Peratt -petroglyphs' - 
/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/corona virus/Howell - corona virus.html:194:
/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/corona virus/Howell - corona virus.html:196:
/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/corona virus/Howell - corona virus.html:204:
>> I either fixed the fname, or re-[copy, paste]ed the fname
?subDir not working? :
/media/bill/SWAPPER/Website - raw/page Software programming.html:56:
