#] 
#] *********************
#] "$d_SysMaint"'webSite/0_website bash notes.txt'  - bash-related webSite maintenance & tracking
www.BillHowell.ca  15May2020 initial
To view this file - use a text editor (not word processor) constant width font (eg courrier 10), tab - 3 spaces


48************************************************48







#24************************24
# Table of Contents, generated with :
# $ grep  "^#]"  "$d_SysMaint"'webSite/0_website bash notes.txt'  |  sed 's/^#\]/  /'  
#
   *********************
   "$d_SysMaint"'webSite/0_website bash notes.txt'
   +-----+
   Instructions : 
   see also notes : 
      bash-related!! : 
      QNial-related : 
   USE lftp for [webSite, ssh] deletion of files not on d_webSite - 
      much slower, hopefully more reliable?
   DON'T use FileZilla for massive stuff!!!! - reloads most of webSite every time!!!!
   OLD or small transfers : To update website : 
   Make sure that upload transfer settings are correct!!!
      Remove remaining transfer queue :
      Re-instate transfer only newer files :
   16Feb2023 externalURL-only approach to links: simpler, safer, faster maintenance      
      comments to come later...
   +-----+
   16Feb2023 executeEmbed examples - old double-step webSite management coding
   11Dec2020 Fixes : confGuides Header - QNial program to do it
   10Dec2020 Fixes : confGuides Menus, webPageSite_update
   10Dec2020 Fix webpage menu links
   04Oct2020 Fix webpage links
   02Oct2020 website update - check all links
   22Sep2020 rsync WebRaw -> Web
   18May2020 Graphs of corona virus by country, grouped by region
   15May2020 Scandanavia corona virus, response to Ian Hepher's Facebook post
   15May2020 Cool email notifications, eg Russ Hall
   24Apr2020 put in for updates of ONLY changed dw_base html files (NYET!)
   23Apr2020 du  --all  --max-depth=2  /home/bill/PROJECTS/System_maintenance
   22Apr2020 update_webPage  IS OP  webPage
   22Apr2020 find  "$d_webRawe"  "file-insert" "*.html" files


48************************************************48
#] +-----+
#] Instructions : 
#] see also notes : 
#]    bash-related!! : 
	"$d_SysMaint"'webSite/0_website bash notes.txt'  - bash-related webSite maintenance & tracking 
	"$d_SysMaint"'webSite/1_website bash upload instructions.txt'  - bash-related webSite instructions
	"$d_SysMaint"'internet & wifi/lftp notes.txt'
	"$d_PROJECTS""bin - secure/lftp update www-BillHowell-ca.sh"  - main tool for webSite updates (not FileZilla)
#]    QNial-related : 
	"$d_Qndfs"'webSite/0_webSite QNial notes.txt'  - QNial webSite notes for maintenance & tracking 
	"$d_Qndfs""webSite/1_webSite QNial process instructions.txt" - fixes for webSite link problems 


#] USE lftp for [webSite, ssh] deletion of files not on d_webSite - 
#]    much slower, hopefully more reliable?
$ bash "$d_PROJECTS""bin - secure/lftp update www-BillHowell-ca.sh"  
>> isn't working well, script problems?
09Dec2020 WARNING!  did my lftp settings pass permissions to the website, 
          denying access to the public for many directories?
          vsftpd - very fast, use this? (secure - don't need for my website, though)
05Jul2022 : 
#] DON'T use FileZilla for massive stuff!!!! - reloads most of webSite every time!!!!
	just use it for very limited transfers - manually-sellected, small directories etc
+-----+
#] OLD or small transfers : To update website : 
1.	rsync PROJECTS -> WebRaw
2.	rsync WebRaw -> Web but xHTML exclusions (doesn't update html files)
3.	update [*.htm, menu] files in WebRaw
4.	QNial "Website updates.ndf" - to update Web/*.html files
5. rsync or FileZilla Web -> http://www.BillHowell.ca
	a) FileZilla check file transfer settings!! - see details next sub-section below
	6.	Add : Sarah, Catherine links
+-----+
#] Make sure that upload transfer settings are correct!!!
	Menu -> View -> Directory listing filters -> check ONLY "Temporary & backup files" for local filters
		  -> Edit filter rules -> "Temporary & backup files" : 
					Filename ends with	:  [~, .bak.]
					Filename contains		:  [References, z_Archive, z_Old, z_References]
					UN-check :	Conditions are case sensitive, 
					check		:	Filter applies to :  Files, Directories
	Click OK to retain changes
#]    Remove remaining transfer queue :
	Menu -> Edit -> Clear private data -> check Clear transfer queue box
	click in transfer [queued files, failed transfers, successful transfers] windows 
		and [clear, delete] lists after all done
#]    Re-instate transfer only newer files :
	Menu -> Edit -> settings -> Transfers -> File exists action :
				Downloads -> Overwrite file if source file newer
				Uploads	 -> Overwrite file if source file newer
#] 16Feb2023 externalURL-only approach to links: simpler, safer, faster maintenance
#]    comments to come later...

## 24************************24
#] +-----+
#] ToDos : 
#] 18Feb2023 process for webSiteLinks update : see 

10Nov2023 setup separate update for ConfGuides to pHtmlConfGuideL.txt





#] +-----+
48************************************************48





08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] ??Nov2023 


08********08
#] 13Nov2023 fix htmlHeadings_to_TblOfContents
example  "$d_web"'Neural nets/callerID-SNNs/callerID-SNNs.html'
after a HUGE amount of [stumble, trip]ing, it now indents cleanly


08********08
#] 11Nov2023 "pStrP Google Analytics 3 tag.txt" failure to correct prodn version?
nyet, that's not the reason
just fix manually  painful but sure...
finished : 
	Neural nets

fix the restart at : 
	"$d_web"'Personal/Challenging the dogmas.html'


08********08
#] 10Nov2023 manually fix: remove Google Analytics 3 header; add GA4 header if necessary 
too much work! I need a script!

<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-4MVH745EPB"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'G-4MVH745EPB');
</script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'UA-136580492-1');
</script>

adapt pOvrL_pStrP_pIns_replaceLines()


08********08
#] 09Nov2023 replace Google Analytics 3 header with GA4 header : 
"$d_bin"'fileops run webSite.sh' :
	# Google Analytics 3 header with GA4 header : 
		pOvL="$d_webWork"'pHtmlPathAll_L.txt'
		pStP="$d_webWork"'pStrP Google Analytics 3 tag.txt'
		pIns="$d_webWork"'pHeader Google Analytics 4 tag.txt'
		pOvrL_pStrP_pIns_replaceLines  "$pOvL"  "$pStP"  "$pIns"

19:09$ bash  "$d_bin"'fileops run webSite.sh'  
strt585 = <!-- Global site tag [(]gtag.js[)] - Google Analytics -->
endd585 = </script>

>> Nuts - I should have a pLog!!
"$d_bin"'bin logs/pOvrL_pStrP_pIns_replaceLines log.txt'
>> Nuts again: every webPage still retains : 
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'UA-136580492-1');
</script>
>> why?  I fixed that for the tests!?!?!?

Some still have ONLY:
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-136580492-1"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'UA-136580492-1');
</script>
>> copy-paste 'pHeader Google Analytics 4 tag.txt'

I stopped for the day going down d_web @ Neural nets...


08********08
#] 28Jul2023 must setup test for povr[,L]_pStrPL_replace
see "$d_bin"'0_test/fileops/povr_strP_replace/0_povr_strP_replace_test notes.txt'


08********08
#] 26Jul2023 one-time convert all "interWebPage links" : 
oops, forgot about letter case : 
change :
	povrL_strP_replace  1  "$pAllL"  'http:\/\/www\.billhowell\.ca\/'  '/home/bill/web/'  
to : 
	povrL_strP_replace  1  "$pAllL"  'http:\/\/www\.BillHowell\.ca\/'  '/home/bill/web/'  
>> BINGO! it worked!

	pLinkL_create()					# create lists of [HREF, IMG] links, ignores intra-webPage links

take out temporarily (already done) : 
http:\/\/www\.billhowell\.ca\/	/home/bill/web/
http:\/\/www\.BillHowell\.ca\/	/home/bill/web/

for pStrPL="$dWebWork"'pStrPL urls interWebPage short-to-longs.txt' : 
geany regexpr 
	search  \.		replace  \\.
	search  \/		replace  \\/
	search  \n		replace  \t"/home/bill/web/



08********08
#] 26Jul2023 revamp "$d_bin"'webSiteLinks update.sh' : pLsts_create()

$ bash  "$d_bin"'webSiteLinks update.sh' 
/home/bill/web/bin/webSiteLinks update.sh: line 310: syntax error: unexpected end of file
>> hard to find problem - check fileops.sh
$ bash "$d_bin"'fileops.sh' 
~
>> That's not a problem?
change :
	source  "$d_bin"'fileops.sh'
to :
source  "$d_bin"'fileops.sh'
>> OK, looks good, but no output : 
$ bash "$d_bin"'fileops.sh' 
~

$ dWebWork="$d_web"'webWork/'
~
$ pHdrFootL="$dWebWork"'urls pL webWork [head, foot].txt'
~
$ find "$dWebWork" -maxdepth 1  -type f         -name "*[head, foot]*.html"     >"$pHdrFootL"
~
>> this works perfectly, why doesn't pLsts_create()?

$ bash  "$d_bin"'webSiteLinks update.sh' 
/home/bill/web/bin/webSiteLinks update.sh: line 161: syntax error: unexpected end of file
>> NUTS!!! back to mysterious error!!

$ dWebWork="$d_web"'webWork/'
~
$ pHdrFootL="$dWebWork"'urls create webWork [head, foot].txt'
~
$ find "$dWebWork" -maxdepth 1  -type f         -name "*[head, foot]*.html"     >"$pHdrFootL"
~
>> again, works perfectly
why doesn't script work?

oops, fNam changes : no longer "*[head, foot]*.html"
$ pHdrFootL="$dWebWork"'urls create [head, foot].txt'
~
$ find "$dWebWork" -maxdepth 1  |  grep  "^head\|footer"        >"$pHdrFootL"
~
>> OK with footers, but no heads?!?!
>> can't use `^ in grep?
>> but now get 'pinnL heads test one.txt' and junk like that
Idiot!! path, not fname : 
$ find "$dWebWork" -maxdepth 1  |  grep  "\/head"       >"$pHdrFootL"
~
>> works great
$ find "$dWebWork" -maxdepth 1  |  grep  "\/head\|\/footer"  |  sort  >"$pHdrFootL"
~
>> even better

$ find "$d_web"  -type f  -name "*.html"  |  grep  --invert-match  "z_Old\|z_Archive\|z_History\|Conference guides\|webWork\|code develop_test\|Cool emails\|References"  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  -i   --with-filename  '<!--  \[#!:'  "FILE"  |  sed 's|\(.*\):<!.*|\1|'  |  sort -u   >"$pNonWebWorkL"
~
>> OK

$ cat "$pHdrFootL"  "$pMenuL"  "$pNonWebWorkL"  |  sort  -u  >"$pAllL"
~
>> OK

$ pNoHomeURLL="$dWebWork"'urls create noHomeURL.txt'
~

$ cat "$pAllL"  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  -i   --with-filename  '<!--  \[#!:'  "FILE"  |  grep    --invert-match  '\/home\/bill\/web\/'  |  sort -u   >"$pNoHomeURLL"
grep: /home/bill/web/economics, markets/Fischer, David 1996 The Great Wave/0_Fischer - The Great pricing Waves 1200-1990 AD.html: <!--  [#!:http://www.BillHowell.ca/webWork files/head_two.html:!#]  -->: No such file or directory
>> oops, nothing in output file, but all paths are OK?

not now - need to check [<A HREF=, IMG SRC=] later : 
	# find those without '/home/bill/web/' 
	cat "$pAllL"  |  grep  --invert-match  "\/home\/bill\/web\/"  >"$pNoHomeURLL"

script wasn'ty working because I inlcuded () in call, example  : 
	pLsts_create()


08********08
#] 24Feb2023 [edit, test, fix] pHtml_cutPutEmbeds()  

function pinnL_line_replace_pLineL()  
   to add '<!--  [#!:end of Embed Sequence:!#]  -->' 
   following : <!--  [#!:http://www.BillHowell.ca/webWork files/head_one.html:!#]  -->
               <!--  [#!:http://www.BillHowell.ca/webWork files/head status.html:!#]  -->
see 'fileops.sh'

#  +-----+
#  ToDos : 
#  23Feb2023 plan forward
#     [edit, test, fix] pHtml_cutPutEmbeds()  - [cut old, insert new] embedExecute html files 
#     webLocal_update() - copy all webPages to webWork so all viewing is local (no backups)
#     webSite_update_lftp()  - lftp to [upload, delete extra] online [file, dir]s   
#  +-----+
#  pLsts_create()  - create file listings 
#  +--------------------+
#  historical-only changes - just used to convert from old QNial style of embedExecutes : 
#  QNialEmbeds_to_pLst()  - extract unique list of embedExecute lines, no args 
#  QNialEmbeds_To_htmlEmbeds()  - convert old embedExecute lines to new htmlEmbeds
#  remove_backtrack()  - remove 'backtrack' from my html files
#  remove_linkError()  - remove '!!linkError!!' from my html files
#  +-----+
#  pStrPL lists - for changing strings in files 
#  create_pStrPL_dirChanges()  - known recent movements of dirs 
#  special pStrpL  - are often created manually
#  combine_pStrLP  - cat [dirChanges, special] pStrPL changes, apply to all my webPages
#  +--------------------+
#  Make changes 
#  urls_update() - combined [dir, special] pStrPL changes, apply to webPages
#  pHtml_cutPutEmbeds()  - [cut old, insert new] embedExecute html files 
#  webLocal_update() - simply copy all webPages to webWork subDir so all viewing is local
#  webSite_update_lftp()  - lftp to [upload, delete extra] online [file, dir]s   

+-----+
olde code 

# for all, copy content :  
# from :  $d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'  
# to   :  $d_web"'webWork/urls pLst test QNialEmbeds_To_htmlEmbeds.txt'  

# +-----+
# 23Feb2023 description of actual iterative process - ad to re-change often, one cycle : 

# +--+
#	"already-altered" files, see 'urls QNialEmbeds pLst [,test] bag.txt' 
#	OK		/home/bill/web/home.html:<!-- start  "http
#	OK		/home/bill/web/Neural nets/Neural Networks.html:<!-- start  "http
#	OK		/home/bill/web/page blogs.html:<!-- start  "http
#	OK		/home/bill/web/page hosted subsites.html:<!-- start  "http
#	OK		/home/bill/web/page projects.html:<!-- start  "http
#	OK		/home/bill/web/page Software programming.html:<!-- start  "http
#	OK		/home/bill/web/Projects - mini/Transformer NNs/0_Transformer NNs.html:<!-- start  "http
#	Make sure that the above do not appear in the general list to update Embeds!?!?

# +--+
#	manually regexpr change "already-altered" files via geany txtEditor : 
#	search :  '<!\-\-[ ]*start[ ]*\"'		replace :  '<!--  \[#!:' <2 spaces>
#	search :  '"[ ]*\-\->'						replace :  ':!#]  -->'
#	geany txtEditor whole document :   
#		replace : '<!-- end -->\n' 
#		with    : '' (null)
#	geany txtEditor insert, @end of embeds [Head_one, normalStatus], but NOT 'footer endFile', 
#		insert '<!--  [#!:end of Embed Sequence:!#]  -->\n'
#	@end of file : NO embed coding (finishes naturally?)

#	"already-altered" files : 
#	$ grep  '\[#!:'  "$d_web"'Neural nets/Neural Networks.html'
#	<!--  [#!:http://www.BillHowell.ca/webWork files/fin Head_one.html:!#]  -->
#	<!--  [#!:end of Embed Sequence:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/fin Head_two.html:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/Menu.html:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/Menu neural nets.html:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/normalStatus.html:!#]  -->
#	<!--  [#!:end of Embed Sequence:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/fin Footer.html:!#]  -->
#	>> YES! finally...
#	But list should be : 
#	<!--  [#!:http://www.BillHowell.ca/webWork files/head_one.html:!#]  -->
#	<!--  [#!:end of Embed Sequence:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/head_two.html:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/Menu.html:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/Menu neural nets.html:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/head status.html:!#]  -->
#	<!--  [#!:end of Embed Sequence:!#]  -->
#	<!--  [#!:http://www.BillHowell.ca/webWork files/footer normal.html!#]  -->

08********08
#] 23Feb2023 update_webLocal - test easy new idea for local viewing of webPages
see "$d_bin"'webSiteLinks update.sh' 
doesn't require separate full dirs

+-----+
olde code

#] pActualLinklDiffL_create_pStrPLDirFname()  -  non-[webWork, conference guides]
# do I need  grep --invert-match  '//'  = '\/\/' below?

	pActualLinklDiffL_create_pStrPLDirFname() 
{

	# directories 
	# pWebDirL 
	find "$d_web" -type d  |  grep  --invert-match  "z_Old\|z_Archive\|Conference guides\|webWork\|Cool emails\|Yoonsuck Choe - conf program book\|Niroma\|Weart\|Qnial/Manuals\|Top 75 Immunotherapy startups\|\/\/"  |  sed 's|/home/bill/web/|http://www.BillHowell.ca/|'  |  sed  's|$|/|'  >>"$d_web"'webWork/webSite dirActualL.txt'


	# pWebLinkL 
	find "$d_web" -type d  |  sed  's|$|/|'  |  grep  --invert-match  "z_Old\|z_Archive\|Conference guides\|webWork\|Cool emails\|Yoonsuck Choe - conf program book\|Niroma\|Weart\|Qnial/Manuals\|Top 75 Immunotherapy startups"  |  grep --invert-match  '//'  |  sed 's|/home/bill/web/|http://www.BillHowell.ca/|'  |  sed 's|\(.*\)Projects - major\/\(.*\)|\1\2\t\1Projects - major\/\2|'  >>"$d_web"'webWork/urls dirChanges pStrPL.txt'

	# links : 


}  



08********08
#] 23Feb2023 pinnL_pStrPL_replace [all webPages, combined changes] (very slow replacements!!)
run bash  "$d_bin"'webSiteLinks update.sh' 
	# 23Feb2023 OK worked 
	date_ddmmmyyyy_hm=$(date +"%e%B%Y %kh%Mm")
	echo "start pinnL_pStrPL_replace : $date_ddmmmyyyy_hm"
	pinnL_pStrPL_replace  1  "$d_web"'webWork/urls non-[conference guides] pLst test bag.txt'  "$d_web"'webWork/230223 15h33m urls combined changes pStrPL.txt' 
	date_ddmmmyyyy_hm=$(date +"%e%B%Y %kh%Mm")
	echo "ended pinnL_pStrPL_replace : $date_ddmmmyyyy_hm"

oops timing ~15:54 to 16:10 ~= 16 minutes for 9 html webPages (>1.6 minutes each, say 2)
so for 87 webPages in my webSite : <= 87*2 = 174 minutes ~3 hours???
start pinnL_pStrPL_replace : 23February2023 16h27m
>> guess 19h30m...
ended pinnL_pStrPL_replace : 23February2023 18h01m
>> closer to 1.5 hours

I really need an error failure if [file, dir] doesn't exist!!
>> I wrote some stuff update_webLocal()

08********08
#] 23Feb2023 compose special pStrPL for [fEmbeds, etc]
see "$d_web"'webWork/230223 13h55m urls special changes pStrPL.txt'  
	/home/bill/web/webWork/230223 13h55m urls special changes pStrPL.txt
fin Head_one.html	head_one.html
fin Head_two.html	head_two.html
normalStatus.html	head status.html
fin Footer.html	footer normal.html

Like dirChanges, these are for : 
	"$d_web"'webWork/urls non-[conference guides] pLst all.txt'

# +-----+
# special manual changes prior to QNialEmbeds work
>> check that [backtrack, !!linkError!!] changes have been done! - output files shoul;d be empty

	# 'backtrack' - find files 
	cat  "$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  --with-filename -i  '\[#=; backtrack ;=#\]'  "FILE"  |  sed  's|\(.*\):.*|\1|'  |  rev  |  sed  's|.*:||;s|\(.*\)\>".*|\1|'  |  rev  |  sort -u  >"$d_web"'webWork/urls pLst backtrack.txt'  
	
	# '!!linkError!!' - find files  
	cat  "$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  --with-filename     '!!linkError!!'  "FILE"  |  sed  's|\(.*\):.*|\1|'  |  rev  |  sed  's|.*:||;s|\(.*\)\>".*|\1|'  |  rev  |  sort -u  >"$d_web"'webWork/urls linkError pLst.txt'  

OOPS - !!linkError!! changes not comlete in : 
OK		/home/bill/web/economics, markets/Fischer, David 1996 The Great Wave/
			0_Fischer - The Great pricing Waves 1200-1990 AD.html
OK		/home/bill/web/economics, markets/market data/SLregress/200912 semi-log/
			1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html
OK		/home/bill/web/Projects - major/Charvatova solar inertial motion & activity/
			_Charvatova - solar inertial motion & activity.html
OK		/home/bill/web/Projects - major/Climate and sun/_Climate and sun.html
OK		/home/bill/web/Projects - major/Pandemics, health, and the Sun/
			_Pandemics, health, and the sun.html

Simply search :  '!!linkError!!'  replace : 'http://www.BillHowell.ca/'
All very simple except the file : 
	'1872-2020 SP500 index, ratio of opening price to semi-log detrended price.html' 
lots of work on the file!! several possibilities, as files slot up : 
	economics, markets/SP500/multi-fractal/	economics, markets/market data/detrend/200913/
	economics, markets/SP500/multi-fractal/	economics, markets/market data/SLregress/200912 semi-log/

08********08
#] 23Feb2023 Do ALL of my webPages now have the current fEmbed coding?
# find those without 'http://www.BillHowell.ca/' 
	cat "$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  -i   --with-filename  '<!--  \[#!:'  "FILE"  |  grep    --invert-match  '<!--  \[#!:http://www.BillHowell.ca/'  |  sed 's|\(.*\):<!.*|\1|'  |  sort -u   >"$d_web"'webWork/urls non-[webWork, conference guides] pLst no homeURL.txt'
results : 
		/home/bill/web/Bill Howells videos/Howell - videos.html
		/home/bill/web/economics, markets/Fischer, David 1996 The Great Wave/
				0_Fischer - The Great pricing Waves 1200-1990 AD.html
		/home/bill/web/economics, markets/PE Schiller forward vs 10yr Tbills/
				S&P 500 Shiller-forward PE versus 10y Treasury bond rates.html
		/home/bill/web/home.html
		/home/bill/web/Neural nets/Neural Networks.html
		/home/bill/web/page blogs.html
		/home/bill/web/page hosted subsites.html
		/home/bill/web/page projects.html
		/home/bill/web/page Publications & reports.html
		/home/bill/web/page Software programming.html
		/home/bill/web/Projects - major/Electric Universe/
				Peratt - Auroral phenomena and petroglyphs.html
		/home/bill/web/Projects - major/Solar modeling and forecasting/
				_Solar modeling & forecasting.html
		/home/bill/web/Projects - mini/Transformer NNs/0_Transformer NNs.html
		/home/bill/web/webSites - other people/Steven H Yaskell/0_Steven H Yaskell.html

But the replacement /home/bill/web/ -> 'http://www.BillHowell.ca' is automatic.
I shouldn't spend time on this except that I need a full lost of my webPages.
>> Nuts, those files have '<!--  [#!:end of Embed Sequence'

	cat "$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  -i   --with-filename  '<!--  \[#!:'  "FILE"  |  grep    --invert-match  '<!--  \[#!:http://www.BillHowell.ca/\|<!--  \[#!:end of Embed Sequence'  |  sed 's|\(.*\):<!.*|\1|'  |  sort -u   >"$d_web"'webWork/urls non-[webWork, conference guides] pLst no homeURL.txt'
>> Now output is empty - but I don't trust it.
There are now 87 webPages in 'urls non-[webWork, conference guides] pLst all.txt'  

next step...


08********08
#] 22Feb2023 insert a code for ALL my webPages, to list pths easily
actually, already there :
	<!-- start  "http://www.BillHowell.ca/webWork files/fin Head_one.html" -->
	<!-- start  "http://www.BillHowell.ca/webWork files/fin Head_two.html" -->
	etc...
in general want :  '<!--  [#!:http://www.BillHowell.ca/[...]':!#]  -->

+-----+
"already-altered" webPages, fEmbeds are present. 
see 'urls QNialEmbeds pLst [,test] bag.txt' 
OK		/home/bill/web/home.html
OK		/home/bill/web/Neural nets/Neural Networks.html
OK		/home/bill/web/page blogs.html
OK		/home/bill/web/page hosted subsites.html
OK		/home/bill/web/page projects.html
OK		/home/bill/web/page Software programming.html
OK		/home/bill/web/Projects - mini/Transformer NNs/0_Transformer NNs.html

23Feb20223 : 

These webPages have proper coding for fEmbeds, but NO fEmbeds are there. 
I added lines '<!--  [#!:end of Embed Sequence:!#]  -->'  :  
OK		/home/bill/web/Projects - major/Electric Universe/
			Peratt - Auroral phenomena and petroglyphs.html
OK		/home/bill/web/Projects - major/Civilisations and sun/_Civilisations and the sun.html
OK		/home/bill/web/page Publications & reports.html
OK		/home/bill/web/page Software programming.html  (..yikes - was STILL wrong!?!?!)
OK		/home/bill/web/economics, markets/PE Schiller forward vs 10yr Tbills/
				S&P 500 Shiller-forward PE versus 10y Treasury bond rates.html
OK		/home/bill/web/economics, markets/Fischer, David 1996 The Great Wave/
				0_Fischer - The Great pricing Waves 1200-1990 AD.html
OK		/home/bill/web/Bill Howells videos/Howell - videos.html
OK		/home/bill/web/Professional & Resume/_Resumes, work experience.html
		/home/bill/web/Projects - mini/Mythology/
				Cardona - gods are planets, planets are gods (plus Sun).html

Make sure that the above do not appear in the general list to update Embeds!?!?
>> NYET! its OK, even if a few extra added of : 
		'<!--  [#!:end of Embed Sequence:!#]  -->\n'

manually regexpr change "already-altered" files via geany txtEditor : 
	regexpr search : '<!\-\-[ ]*start[ ]*\"'	replace :  '<!--  \[#!:' <2 spaces>
	regexpr search : '"[ ]*\-\->'					replace :  ':!#]  -->'
	escOnly search : '<!-- end -->\n'			replace : '' (null)
	geany txtEditor insert, @end of embeds [Head_one, normalStatus], but NOT 'footer endFile', 
		insert '<!--  [#!:end of Embed Sequence:!#]  -->\n'
	@end of file : NO embed coding (finishes naturally?)

$ grep  '<!\-\-[ ]*start[ ]*\"'  "$d_web"'Neural nets/Neural Networks.html'
>> nothing
$ grep  "<\!--[ ]*start[ ]*\""  "$d_web"'Neural nets/Neural Networks.html'
bash: !-: event not found
$ grep  "<\!--  \[#!:"  "$d_web"'Neural nets/Neural Networks.html'
bash: ": unrecognized history modifier
$ grep  "\[#!:"  "$d_web"'Neural nets/Neural Networks.html'
bash: ": unrecognized history modifier
$ grep  '[#!:'  "$d_web"'Neural nets/Neural Networks.html'
grep: Unmatched [, [^, [:, [., or [=
$ grep  'Bill'  "$d_web"'Neural nets/Neural Networks.html'
>> OK, lots!
$ grep  '\[#!:'  "$d_web"'Neural nets/Neural Networks.html'
<!--  [#!:http://www.BillHowell.ca/webWork files/fin Head_one.html:!#]  -->
<!--  [#!:end of Embed Sequence:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/fin Head_two.html:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/Menu.html:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/Menu neural nets.html:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/normalStatus.html:!#]  -->
<!--  [#!:end of Embed Sequence:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/fin Footer.html:!#]  -->
>> YES! finally...
But list should be : 
<!--  [#!:http://www.BillHowell.ca/webWork files/head_one.html:!#]  -->
<!--  [#!:end of Embed Sequence:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/head_two.html:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/Menu.html:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/Menu neural nets.html:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/head status.html:!#]  -->
<!--  [#!:end of Embed Sequence:!#]  -->
<!--  [#!:http://www.BillHowell.ca/webWork files/footer normal.html!#]  -->


+-----+
webPages with : 
<!--- :&file-insert &: "fin Head_one.html" --->		
<!--- :&insert-ender&: --->		
	<TITLE>Howell web - Solar modelling and forecasting</TITLE>
<!--- :&file-insert &: "fin Head_two.html" --->		
<!--- :&insert-ender&: --->
<!--- :&file-insert &: "Menu1.html" --->			
<!--- :&insert-ender&: --->			
<!--- :&file-insert &: "Menu projects1.html" --->	
<!--- :&insert-ender&: --->
<!--- :&file-insert &: "fin Footer1.html" --->
<!--- :&insert-ender&: --->

this only involves :
		/home/bill/web/Projects - major/Solar modeling and forecasting/
				_Solar modeling & forecasting.html
		/home/bill/web/webSites - other people/Steven H Yaskell/0_Steven H Yaskell.html

manually regexpr change "already-altered" files via geany txtEditor : 
	regexpr search  : '<!\-\-\-[ ]*:&file-insert &: \"'		
			  replace : '<!--  \[#!:http://www.BillHowell.ca/webWork files/' <2 spaces>
	regexpr search : '"[ ]*\-\-\->'									replace :  ':!#]  -->'
	escOnly search : '<!--- :&insert-ender&: --->\n'		replace : '' (null)
	geany txtEditor insert, @end of embeds [Head_one, normalStatus], but NOT 'footer endFile', 
		insert line '<!--  [#!:end of Embed Sequence:!#]  -->'
	@end of file : NO embed coding (finishes naturally?)
	add subDirs 'http://www.BillHowell.ca/' - were missing!!...

+-----+
olde code

	# 22Feb2023 I need to insert a code for ALL my webPages, without which they are NOT included!
	# actually, alreay there 
	find "$d_web"  -type f  -name "*.html"  |  grep  --invert-match  "z_Old\|z_Archive\|Conference guides\|webWork\|code develop_test\|Cool emails\|Yoonsuck Choe\|Niroma\|Weart\|Qnial/Manuals\|Top 75 Immunotherapy startups    \|Randi Foundation   \|version-for-dvisvgm\|References"  |  sort -u   >>"$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'
	
	cat "$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'    |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  --with-filename  -i  'BillHowell'  "FILE"  |  sed 's|\(.*\):.*|\1|'  |  sort -u   >>"$d_web"'webWork/urls non-[webWork, conference guides] pLst all Howell.txt'

only returns first two lines but NOT the filename : 
	find "$d_web"  -type f  -name "*.html"  |  grep  --invert-match  "z_Old\|z_Archive\|Conference guides\|webWork\|code develop_test\|Cool emails\|Yoonsuck Choe - conf program book\|Niroma\|Weart\|Qnial/Manuals\|Top 75 Immunotherapy startups\|Randi Foundation\|version-for-dvisvgm\|bogoTrainer.html\|References/Neural Nets\|References/Charbonneau\|References/110313 ABC news - tsunami"  |  tr  \\n  \\0  |  xargs -0  -IFILE  head -n 2  "FILE"  |  grep  -i  '<!\-\- start  '  |  sed 's|\(.*\):.*|\1|'  |  sort -u   >"$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'

	#	19Feb2023 still includes references and some material that are not Howell webPages : 
	# my webPages all have '<!\-\-  \[#!:' -> I would need to use xargs!!
	
	# I need to insert a code for ALL my webPages, without which they are NOT included!
	find "$d_web"  -type f  -name "*.html"  |  grep  --invert-match  "z_Old\|z_Archive\|Conference guides\|webWork\|code develop_test\|Cool emails\|Yoonsuck Choe - conf program book\|Niroma\|Weart\|Qnial/Manuals\|Top 75 Immunotherapy startups\|Randi Foundation\|version-for-dvisvgm\|bogoTrainer.html\|References"  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  -i  '<!-- start  '  "FILE"  |  sed 's|\(.*\):.*|\1|'  |  sort -u   >"$d_web"'webWork/urls non-[webWork, conference guides] pLst all.txt'


08********08
#] 20Feb2023 [code, test, run] pHtml_cutPutEmbeds

first edit instructions in "$d_bin"'webSiteLinks update.sh' 
for pHtml_cutPutEmbeds - need combined [, non-]webWork list
add to pLst_base() : 
	cat "$d_web"'webWork files/urls pLst base webWork [head, foot].txt'  "$d_web"'webWork files/urls pLst base webWork Menu.txt'  "$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'  >"$d_web"'webWork files/urls pLst base non-[conference guides].txt'

in create_pLFnamDirLinkL()  - create files listing [fname, dir, link]s in non-conference html
compare :  
file#
94		# above result			: "$d_web"'webWork files/webLink intern diffL.txt' 
61		# manually-prepared  : "$d_web"'webWork/urls pStrPL dirChanges.txt'

Just over half are comparable, leaving 30+ subDirs requiring manual.
Just proceed with "$d_web"'webWork/urls pStrPL dirChanges.txt' : 
	it's cleaner (not so much junk)
	safer - have to have a "middle subDir", asd one-ended prone to [mass, double changes] etc

Do I need to do this?
Add backslashes to "$d_web"'webWork/urls pStrPL dirChanges.txt'
save changes    to "$d_web"'webWork/urls pStrPL dirChanges backslashes.txt'

Fix problem next, before I do dirChanges : 
!!linkError!!
!!linkError!!Bill Howells videos/120214 Venus et Mars, au dela d une histoire d amour/
!!linkError!!Bill Howells videos/140214 Nazis saved Norwegians video/
!!linkError!!Charvatova solar inertial motion & activity/Verification/
!!linkError!!Climate and sun/
!!linkError!!Cool emails/
!!linkError!!economics, markets/SP500/multi-fractal/
!!linkError!!LibreOffice/
!!linkError!!/national/nationalpost/
!!linkError!!/national/nationalpost/search/
!!linkError!!Pandemics, health, and the Sun/
!!linkError!!Pandemics, health, and the Sun/corona virus/Howell - corona virus.html#Cosmic/
!!linkError!!Pandemics, health, and the Sun/corona virus/Howell - corona virus.html#New corona virus cases/day/
!!linkError!!Paul L Vaughan/
!!linkError!!Personal/
!!linkError!!SP500 1928-2020 yahoo finance.dat
!!linkError!!Table of Contents
>> change '!!linkError!!'  to  nothing ('http:\/\/www.BillHowell.ca\/' already unaltered)

geany 
regexpr	search :	(\!\!linkError\!\!)(.*)		replace : \1\2\t\2
	http:\/\/www.BillHowell.ca\/


-----+
olde code

# webSite_update()  - do it all for real AFTER thorough testing!!
# 18Feb2023 temporary - first and only time 

   webSite_update()
{  
	# first time only, MUST run this first! 
	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst base non-conference guides.txt'  "$d_web"'webWork files/urls web pStrPL backtrack.txt'  

	# first time only MUST change webWork first because Menus are used in webPages 
	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'  "$d_web"'webWork files/urls pStrPL executeEmbed_replaceOldCode.txt'

	# first time only - inserts [header, footer, Menu, etc] files after embedExecute lines 
	menu_updates  1  "$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'  

	# dirChanges, whenever they are done
	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst base non-conference guides.txt'  "$d_web"'webWork files/urls pStrPL dirChanges.txt'  

	# ongoing - to develop later
	# [fnamem dir] changes
	#pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst base non-conference guides.txt'  "$d_web"'webWork files/urls web pStrPL update.txt'  
}  

# webSite_tests()  - manual changes during tests, especially change pLsts, can run from CLI
   webSite_tests()
{  
	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst test non-conference guides.txt'  "$d_web"'webWork files/urls pStrPL backtrack.txt'  

	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst test webWork [header, footer].txt'  "$d_web"'webWork files/urls pStrPL dirChanges.txt'  

	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst test webWork Menu.txt'  "$d_web"'webWork files/urls pStrPL dirChanges.txt'  

	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst test non-[webWork, conference guides].txt'  "$d_web"'webWork files/'  

	pinnL_pStrPL_replace  1  "$d_web"'webWork files/urls pLst test non-[webWork, conference guides].txt'  "$d_web"'webWork files/urls pStrPL executeEmbed_replaceOldCode.txt'  

	executeEmbed_update   1  "$d_web"'webWork files/urls pLst test non-[webWork, conference guides].txt'   
}  


	# |  rev  |  sed  's|.*\/\(.*\)|\1|i'  |  rev  
#	cat  "$d_web"'webWork files/webLink intern pthL.txt'  |  sed  's|http:\/\/www\.BillHowell\.ca\/||;s|\(.*\)\/.*|\1/|'  |  rev  |  sort -u   >"$d_web"'webWork files/webLink intern subDirL.txt' 

#	cat  "$d_web"'webWork files/webLink intern pthL.txt'  |  rev  |  sed  's|.*\/\(.*\)|/\1|'  |  rev  |  sort -u   >"$d_web"'webWork files/webLink intern fNameL.txt' 

# 20Feb2023 do combined urls pStrPL changes of  [dirChanges, linkError]?  
#		NO!! - first do linkerrors, then dirChanges
#		might have to backslash all special characters in pStrPL?
#	

# 20Feb2023 need to test first!!!!?!?


08********08
#] 19Feb2023 [code, test, run] * [QNialEmbeds_To_htmlEmbeds, pHtml_cutPutEmbeds]

+-----
"$d_web"'webWork files/urls pLst base webWork [head, foot].txt'
>> html-only NOT extracted!
"$d_bin"'webSiteLinks update.sh'
>> fix pLst_base()
'find "$d_web" -maxdepth 6  -type f  -o -name "*.html"  
	was had -o for webWork subLsts - one worked, the other didn't??
>> OK, now OK

list ALL of Howell's html files
	cat  "$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  --count --with-filename  'Howell'  "FILE"  |  sort  -u  >"$d_web"'webWork files/urls pLst base non-[webWork, conference guides] Howell.txt'
>> Oops, 'Conference guides' still included!!!

rerun on 'urls pLst base non-[webWork, conference guides].txt'
	find "$d_web" -maxdepth 6  -type f  -name "*.html"  |  grep  --invert-match  "z_Old"  |  grep  --invert-match  "z_Archive"  |  grep  --invert-match  'Conference guides'  |  grep  --invert-match  "webWork"  >"$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'
>> still a problem.  WHY?
>> doesn't work for webWork etc... but does work for [z_Old, z_Archive, 
>> Also should exclude 'Cool emails' 

Maybe the problem is the combo of '--count --with-filename '?  
could use \| in a single invert-match?
#] multiple search terms, --invert-match 
#] \|        repeated patterns - example from link  d_Qndfs 'webSite header.ndf'  :  
#] host link  'find  "'  d_webRawe  '"  -type f  -name "*"  |  grep  --invert-match "z_Old\|z_Archive
#]    \|code develop_test\|System_maintenance\|Qnial_bag\|Cool emails/\|Electric Universe/References/
#]    \|Electric Universe/References/\|Yoonsuck Choe - conf program book/\|fin Head\|Menu\|fin [F,f]ooter
#]    \|fin organisations\|i9018xtp.default/extensions/\|[0-9]\{6\} [0-9]\{2\}h[0-9]\{2\}m[0-9]\{2\}s "  
#]    |  sort >"'  p_nonuniqueFileList  '"  '  ;  

pLst_base() go with : 
	find "$d_web" -maxdepth 6  -type f  -name "*.html"  |  grep  --invert-match  "z_Old\|z_Archive\|Conference guides\|webWork\|Cool emails"  >"$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'

	# Howell's non-[webWork, conference guides] webPages only!
	cat  "$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  --count --with-filename  'Howell'  "FILE"  |  sort  -u  >"$d_web"'webWork files/urls pLst base non-[webWork, conference guides] Howell.txt'
>> OK, by using \| I can get them all (I had already run into this in the past)  

Conference guides

+-----+
olde code 

I removed as not immediately useful : 
# is this useful when dealing with links? atleast it provides all html files 
	find "$d_web" -maxdepth 6  -type f  -name "*.html"  |  grep  --invert-match  "z_Old"  |  grep  --invert-match  "z_Archive"  |  grep  --invert-match  'Conference guides'  >"$d_web"'webWork files/urls pLst base non-conference guides.txt'

	# Howell's non-[webWork, conference guides] webPages only! - not useful, same #files as above
#	cat  "$d_web"'webWork files/urls pLst base non-[webWork, conference guides].txt'  |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  --count --with-filename  'Howell'  "FILE"  |  sort  -u  >"$d_web"'webWork files/urls pLst base non-[webWork, conference guides] Howell.txt'


08********08
#] 17Feb2023 pinnL_pStrPL_replace - try an example file as a check before doing all

	pinnL_pStrPL_replace  1  '/home/bill/web/webWork files/urls web ex-[webWork, z_Archive, z_Old] pLst.txt'  '/home/bill/web/webWork files/urls web ex-[webWork, z_Archive, z_Old] - pStrPL.txt'
test file : 
	/home/bill/web/Qnial/MY_NDFS/email/email Thunderbird - Base64 Encode and Decode Base64 Files, instructions.html
>> nyet - nothing to change
	/home/bill/web/Projects - major/Electric Universe/Peratt - Auroral phenomena and petroglyphs.html

$ bash  "$d_bin"'fileops calls.sh'  
/home/bill/web/webWork files/urls web ex-[webWork, z_Archive, z_Old] pLst.txt
/home/bill/web/webWork files/urls web ex-[webWork, z_Archive, z_Old] - pStrPL.txt
/home/bill/web/bin/fileops.sh: line 811: d_temppinnL_pStrPL_replace 1.txt: No such file or directory
>> I fixed a few bugs

$ bash  "$d_bin"'fileops calls.sh'  
/home/bill/web/webWork files/urls web ex-[webWork, z_Archive, z_Old] pLst.txt
/home/bill/web/webWork files/urls web ex-[webWork, z_Archive, z_Old] - pStrPL.txt
/home/bill/web/bin/fileops.sh: line 811: /media/bill/ramdisk/pinnL_pStrPL_replace temp.txt: No such file or directory
>> OK, except error in pLst that was fixed [manually, in pLst]

Create list of dirChanges in : 
	'urls web ex-[webWork, z_Archive, z_Old] pStrPL dirChanges.txt'

$ find '/home/bill/web/Projects - major/' -maxdepth 1  -type d  |  sed  's|$|/|'  |  grep  --invert-match  'z_Archive'  |  grep --invert-match  'z_Old'  |  grep --invert-match  '//'  |  sed 's|/home/bill/web/|http://www.BillHowell.ca/|'  |  sed 's|\(.*\)Projects - major\/\(.*\)|\1\2\t\1Projects - major\/\2|'  

put into 'urls web ex-[webWork, z_Archive, z_Old] pStrPL dirChanges.txt' 
		where '/home/bill/web/' changed to 'http://www.BillHowell.ca/' 

continue with Proj-mini
$ find '/home/bill/web/Projects - mini/' -maxdepth 1  -type d  |  sed  's|$|/|'  |  grep  --invert-match  'z_Archive'  |  grep --invert-match  'z_Old'  |  grep --invert-match  '//'  |  sed 's|/home/bill/web/|http://www.BillHowell.ca/|'  |  sed 's|\(.*\)Projects - mini\/\(.*\)|\1\2\t\1Projects - mini\/\2|'  

continue with 'webSites - other people'
$ find '/home/bill/web/webSites - other people/' -maxdepth 1  -type d  |  sed  's|$|/|'  |  grep  --invert-match  'z_Archive'  |  grep --invert-match  'z_Old'  |  grep --invert-match  '//'  |  sed 's|/home/bill/web/|http://www.BillHowell.ca/|'  |  sed 's|\(.*\)webSites - other people\/\(.*\)|\1\2\t\1webSites - other people\/\2|'  

Personal '/home/bill/web/Personal'
$ find '/home/bill/web/Personal/' -maxdepth 1  -type d  |  sed  's|$|/|'  |  grep  --invert-match  'z_Archive'  |  grep --invert-match  'z_Old'  |  grep --invert-match  '//'  |  sed 's|/home/bill/web/|http://www.BillHowell.ca/|'  |  sed 's|\(.*\)Personal\/\(.*\)|\1\2\t\1Personal\/\2|'  

[eGroupWare, Python] - where did I have these? find on webSite?
/home/bill/web/Software programming & code/eGroupWare
>> worry about this later...

So far, I have 60 subDirs to update
There are 397 non-[webWork, z_Archive, z_Old] html files 
qnial> 60*397
23820 corrections, almost all in ram memory

After that, I still have to do pExecuteEmbed changes!!


08********08
#] 17Feb2023 generate a listing of all html files EXCEPT dirs [webWork, z_Archive, z_Old] 
and maybe others

$ find "$d_web" -maxdepth 6  -type f  -name "*.html"  |  grep  --invert-match  "z_Old"  |  grep  --invert-match  "z_Archive"  |  grep  --invert-match  "webWork"  >"$d_web"'webWork files/urls web ex-[webWork, z_Archive, z_Old] pLst.txt'

N08********08
#] 17Feb2023 fix all html files in "$d_web"'webWork/'
string replacements 
[#=; backtrack ;=#]	http://www.BillHowell.ca/
"$d_bin"'fileops calls.sh' : 
	pinnL_pStrPL_replace  1  'http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pLst.txt'  'http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pStrPL.txt'
>> this was done one-at-a-time, don't redo

	pinnL_pStrPL_replace  1  'http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pLst.txt'  'http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pStrPL.txt'

...wait until is fix one-at-a-time Menu processing problem...
now do 'Menu earth, sun, astro, history.html' 
>> great, it works!

now try all Menus
>> YEAH!  It all looks good!!!


08********08
#] 17Feb2023 fix one-at-a-time an html file in "$d_web"'webWork/' (test)
'urls webWork Menu - pStrPL.txt' 
'urls webWork Menu - pLst.txt'	start with only one file in 'urls webWork Menu - pLst.txt'
	'/home/bill/web/webWork files/Menu blogs.html'
>> bad example file as it's OK

'Menu crazy themes and stories.html' - has [#=; backtrack ;=#]
so current pStrPL file is OK : [#=; backtrack ;=#]	http://www.BillHowell.ca/
I backed up pLst 
	pinnL_pStrPL_replace  1  'http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pLst.txt'  'http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pStrPL.txt'
$ bash  "$d_bin"'fileops calls.sh'  
http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pLst.txt
http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pStrPL.txt
/home/bill/web/bin/fileops.sh: line 810: http://www.BillHowell.ca/webWork files/urls webWork [head, foot] - pLst.txt: No such file or directory
>> what??? line 810 is 'flagBackup="$1"'  
>> oops, I had 'http://www.BillHowell.ca' instead of '/home/bill/web'

$ bash  "$d_bin"'fileops calls.sh'  
/home/bill/web/webWork files/urls webWork [head, foot] - pLst.txt
/home/bill/web/webWork files/urls webWork [head, foot] - pStrPL.txt
/home/bill/web/webWork files/fin Footer.html
[#=; backtrack ;=#]	http://www.BillHowell.ca
...
This is the [head, foot] pLst, not the Menu list!
>> stupid, I am getting tired

$ bash  "$d_bin"'fileops calls.sh'  
/home/bill/web/webWork files/urls webWork Menu - pLst.txt
/home/bill/web/webWork files/urls webWork Menu - pStrPL.txt
/home/bill/web/webWork files/Menu crazy themes and stories.html
[#=; backtrack ;=#]	http://www.BillHowell.ca
>> backtrack was NOT replaced?! - OK, I had un-commented that line

Again try :
>> ouch!, multiple rlacements :
<http://www.BillHowell.cahttp://www.BillHowell.cahttp://www.BillHowell.calehttp://www.BillHowell.cahttp://www.BillHowell.cashttp://www.BillHowell.caylehttp://www.BillHowell.ca'fonhttp://www.BillHowell.ca-size:10phttp://www.BillHowell.ca'

put in IFS='' before the read!!
'printf '%s\n'  "$line" ' instead of echo 
>> still the same horrible problem

change 'sed  's|.*\x9||' to 's|\t.*||'
>> still the same horrible problem

Maybe the problem is IFS=''?
forgot the -r option for read?
>> same

oops, in : pinn_strOld_strNew_replace  "$flagBackup"  "$pinn384"  "$strOld384"  "$strNew384"  
change pinn384 to pinL384
>> no changes to file at all?

"$pinnL384" -> "$pinn384"
$ bash  "$d_bin"'fileops calls.sh'  
/home/bill/web/bin/fileops calls.sh: line 31: pinnL_pStrPL_replace: command not found
>> Wow! this is radically off-base, 'fileops calls.sh' not corrupted, though?
>> Crap: 'urls webWork Menu - pLst.txt' had been corrupted!
/home/http://www.BillHowell.caill/wehttp://www.BillHowell.ca/wehttp://www.BillHowell.caWohttp://www.BillHowell.cahttp://www.BillHowell.cahttp://www.BillHowell.cafiles/Menuhttp://www.BillHowell.cahttp://www.BillHowell.cahttp://www.BillHowell.cahttp://www.BillHowell.cazyhttp://www.BillHowell.cahttp://www.BillHowell.cahemeshttp://www.BillHowell.cahttp://www.BillHowell.candhttp://www.BillHowell.cashttp://www.BillHowell.caohttp://www.BillHowell.caies.hhttp://www.BillHowell.caml

Problem: cannot put tab into sedExpr input : change 'sed  's|.*\x9||' to 's|\t.*||'?
>> but simple examples worked?

try a simple one- file 
	pinn_strOld_strNew_replace  1  '/home/bill/web/webWork files/Menu crazy themes and stories.html'  '[#=; backtrack ;=#]'  'http://www.BillHowell.ca/webWork files/'
>> oops, MUST escape [], and for menus, only use 'http://www.BillHowell.ca/'

	pinn_strOld_strNew_replace  1  '/home/bill/web/webWork files/Menu crazy themes and stories.html'  '[#=; backtrack ;=#]'  'http://www.BillHowell.ca/'
>> BINGO!
try multiple pLstL now...

08********08
#] 16Feb2023 test str_inWebPages_pout() in "$d_bin"'fileops.sh'
several vexing old problems re-fixed : 
see "$d_SysMaint"'Linux/[bash, echo, find] notes.txt'

08********08
#] 16Feb2023 executeEmbed examples - old double-step webSite management coding
[#!: full-line executeEmbeds,  phraseValueList = (("fout  fout)("backtrack  backtrack))
[#!: path_insertIn_fHand					(link  d_webWork  'fin Head_one.html')		fout  ;  
<TITLE> Publications and reports </TITLE>    
[#!: path_insertIn_fHand					(link  d_webWork  'fin Head_two.html')		fout  ;  
[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  'Menu.html')  phraseValueList  ;  
[#!: path_insertIn_fHand					(link  d_webWork  'normalStatus.html'	)  fout  ;  
[#!: full-line executeEmbeds,  phraseValueList = (("fout  fout)("backtrack  backtrack))
[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  'fin Footer.html')  phraseValueList  ;  

$ find "$d_web"'webWork files/' -type f  -name "*.html"  -maxdepth 1  |  grep  --invert-match  "z_Old"  |  grep  --invert-match  "z_Archive"  |  sort  
/home/bill/web/webWork files/confStatus.html
/home/bill/web/webWork files/fin Footer.html
/home/bill/web/webWork files/fin footer Neil Howell.html
/home/bill/web/webWork files/fin footer Paul Vauhan.html
/home/bill/web/webWork files/fin footer Steven Wickson.html
/home/bill/web/webWork files/fin footer Steven Yaskell.html
/home/bill/web/webWork files/fin Head_one.html
/home/bill/web/webWork files/fin Head_two.html
/home/bill/web/webWork files/fin organisations.html
/home/bill/web/webWork files/Menu blogs.html
/home/bill/web/webWork files/Menu crazy themes and stories.html
/home/bill/web/webWork files/Menu earth, sun, astro, history.html
/home/bill/web/webWork files/Menu hosted subsites.html
/home/bill/web/webWork files/Menu Howell videos.html
/home/bill/web/webWork files/Menu.html
/home/bill/web/webWork files/Menu Lies, Damned Lies, and Scientists.html
/home/bill/web/webWork files/Menu neural nets.html
/home/bill/web/webWork files/Menu professional and resume.html
/home/bill/web/webWork files/Menu projects.html
/home/bill/web/webWork files/Menu software programming.html
/home/bill/web/webWork files/normalStatus.html

do NOT touch : 
/home/bill/web/webWork files/confFoot_authors.html
/home/bill/web/webWork files/confFoot.html
/home/bill/web/webWork files/confHead.html

 
08********08
#] 11Dec2020 Fixes : confGuides Header - QNial program to do it

test : 
# confGuide_header  (link  d_webRawe  'Neural nets/Conference guides/Conference guides.html')  

		subMenu Publicity Guide 
			link from confGuide - OK	
			text under menu is too large
			links in menu -  ALL OK, except "IEEE-CIS ListServers" 
			link "IEEE-CIS ListServers" - screwed up menu, header, has official due date

link "IEEE-CIS ListServers" - screwed up menu, header, has official due date
>> I fixed this - didn't have confGuideHeader
# confGuide_header  (link  d_webRawe  'Neural nets/Conference guides/Publicity website/IEEE-CIS ListServers.html')  
# webPageSite_update  l  (link  d_webRawe  'Neural nets/Conference guides/Publicity website/IEEE-CIS ListServers.html')  
>> looks great

text under menu is too large
>> just run [confGuide_header, webPageSite_update] and sat I get
# testPage :=  link  d_webRawe  'Neural nets/Conference guides/Publicity website/Publicity Guide.html'  
# confGuide_header		 testPage  
# webPageSite_update  l  testPage  
>> looks great

		subMenu Reviewers' Guide 
			link from confGuide - OK					
			NO menu!
			No text comment under menu
>> can't see a obvious problem 
# testPage :=  link  d_webRawe  'Neural nets/Conference guides/Reviewers website/Reviewers guide.html'  
# confGuide_header		 testPage  
# webPageSite_update  l  testPage  
>> looks great

fix column widths of confGuide overall men match other menus
>> can't see any obvious difs - leave for later... much later.

		subMenu Publications Guide 
			link from confGuide - OK
			check ONLY "chair links" in menu			- ALL WRONG!!!, probably same fnames?
			links in "non-Author actions" sub-menu - ALL WRONG!!!, probably same fnames?

[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  'confMenu_publications.html')  phraseValueList  ;
[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  'confMenu_authors.html')  phraseValueList  ;

Now to : 
1.	htmlConfGuidePages - to make it easy to target recent changes to [menu, head, foot]s
2.	webSite_extract_pathsSubDirsFnames - update the "permanent" files
3.	change confGuide headers - change Publications menu to Authors' & Publish menu
4.	EACH confGuide_header  htmlConfGuidePages - random check of results, especially previous problem webPages
5.	webSite_doAll
qnial> update webSite_extract_pathsSubDirsFnames

3.	change confGuide headers - change Publications menu to Authors' & Publish menu
qnial> str_replaceIn_pathList  l  d_webRawe  (link  '[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  '  chr_apo  'confMenu_publications.html'  chr_apo  ')  phraseValueList  ;')  (link  '[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  '  chr_apo  'confMenu_authors.html'  chr_apo  ')  phraseValueList  ;')  htmlConfGuidePages  
>> seems OK (I only  looked at 'pubChair guide.html')

4.	EACH confGuide_header  htmlConfGuidePages - random check of results, especially previous problem webPages
qnial> EACH confGuide_header  htmlConfGuidePages 
>> ran OK, check after next step

5.	webSite_doAll
>> looks great!
>> just a couple of problems popped up : 
Author menu -> IEEE CrossCheck - text-similarity chair link doesn't work
	changed  subDir 'Author guide website'  to  'Publications website'
Oops - Sponsors menu links don't work

Re-run : 
qnial> webSite_doAll



08********08
#] 10Dec2020 Fixes : confGuides Menus, webPageSite_update

confGuides Menus 
webPageSite_update - test a few webPages

qnial> webPageSite_update  l  (link  d_webRawe  'Neural nets/Conference guides/Conference guides.html')  
>> didn't work - looks like the problem is : 

Wait a minute, Change :
+.....+
[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  'confMenu_overall.html)  phraseValueList  ;  
+.....+
To :
+.....+
[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  'confMenu_overall.html')  phraseValueList  ;  
+.....+
>> OK, works!
Change color of confGuide overall menu - light green
same dimensions as rest of my website menus 

Add to confGuide webPages :
[#!: path_insertIn_fHand					(link  d_webWork  'confStatus.html')  fout  ;    

Insert this into ALL confGuide webPages : 
# just test with one file 
	str_replaceIn_pathList  o  d_webRawe  '<TITLE> [Author, Committee]s guides to [IEEE-CIS, INNS] conferences </title> '  (link  '<TITLE> [Author, Committee]s guides to [IEEE-CIS, INNS] conferences </title> '  '[#!: path_insertIn_fHand		'(link  d_webWork  '  chr_apo  'fin Head_two.html'  chr_apo  ')  fout  ;  ')    chr_newline  '<TITLE> Howell : [#=; fname ;=#] </title> '  chr_newline  '[#!: path_insertIn_fHand'  chr_tab  chr_tab  chr_tab  chr_tab  chr_tab  '(link  d_webWork  '  chr_apo  'fin Head_two.html'  chr_apo  ')  fout  ;  ')  (solitary  (link  d_webRawe 'Neural nets/Conference guides/Author guide website/Author guide.html'))  


[#!: path_insertIn_fHand					(link  d_webWork  '')  fout  ;  
[#!: path_executeEmbedsInsertIn_fHand  (link  d_webWork  'confMenu_overall.html')  phraseValueList  ;


08********08
#] 10Dec2020 Fix webpage menu links

+-----+
Problems in d_webSite : 

Menu checks were ONLY done on a "click-through-menu" basis, 
	i.e. don't check all menu links on all webPages with the menu
subMenus were NOT checked, except for "Conference Guides"

Menu main 	ALL OK

Menu Neural Nets	
	row Neural Nets			- ALL OK

	row Conference Guides	
		- no menu at top!?!?
		- all menus EXCEPT [Author, Sponsor]s' Guides should have menu confGuide??
			Perhaps even the Authors should see this?  (another year, perhaps)

		subMenu Authors' Guide
			link from confGuide - OK	
			links in menu -  ALL OK
			links in "non-Author actions" sub-menu - ALL OK	

		subMenu Publications Guide 
			link from confGuide - OK
			check ONLY "chair links" in menu			- ALL WRONG!!!, probably same fnames?
			links in "non-Author actions" sub-menu - ALL WRONG!!!, probably same fnames?

		subMenu Publicity Guide 
			link from confGuide - OK	
			text under menu is too large
			links in menu -  ALL OK, except "IEEE-CIS ListServers" 
			link "IEEE-CIS ListServers" - screwed up menu, header, has official due date

		subMenu Reviewers' Guide 
			link from confGuide - OK					
			NO menu!
			No text comment under menu

		subMenu Sponsors' Guide 
			link from confGuide - OK					
			links in menu -  none work at all
			
+-----+
Problems in webSiteOnLine :  THESE ARE THE SAME AS FOR d_webSite!!!


Simply follow same checks as for d_webSite (above)

Menu checks were ONLY done on a "click-through-menu" basis, 
	i.e. don't check all menu links on all confGuide webPages with the overall confGuide menu

Menu main 	ALL OK

Menu Neural Nets	
	row Neural Nets			
		- two menu links gave : 403 Forbidden "You don't have permission to access this resource." 
			"NN earlier work", "Paper reviews "  
		- "Holidays : NNs & genomics" - is one whole-page-link-mess


	row Conference Guides	
		ALL the same errors as for d_webSite??
		- no menu at top!?!?
		- all menus EXCEPT [Author, Sponsor]s' Guides should have menu confGuide??

		subMenu Authors' Guide
			link from confGuide - OK	
			links in menu -  ALL OK
			links in "non-Author actions" sub-menu - ALL OK	

		subMenu Publications Guide 
			link from confGuide - OK
			check ONLY "chair links" in menu						- ALL WRONG!!!, probably same fnames?
			"chair links" in "non-Author actions" sub-menu	- 1st row WRONG!!!, probably same fnames?

		subMenu Publicity Guide 
			link from confGuide - OK	
			text under menu is too large
			links in menu -  ALL OK, except "IEEE-CIS ListServers" 
			link "IEEE-CIS ListServers" - screwed up menu, header, has official due date

		subMenu Reviewers' Guide 
			link from confGuide - OK					
			NO menu!
			No text comment under menu

		subMenu Sponsors' Guide 
			link from confGuide - OK					
			links in menu -  none work at all


08********08
#] 04Oct2020 Fix webpage links
1.	replace %20 with space for all EXCEPT maybe mailtos
2.	get rid of absolute links to my website (eg http://www.BillHowell.ca ...)
		it is a pain to use relative addressing way up & down directories
		absolute addressing is far more robust
		however - I can't properly work in webRaw with absolute addressing
		04Oct2020 there are ~150 of such absolute links to my web-page


intern links - start with :
	many of the links have "trailing text" because of greedy sed 
		so I can fix that in QNial with 'find'
	As predicted, the [./, ../] backtracking isn't working (was commented out in QNial)
+-----+
/media/bill/SWAPPER/Website - raw/Civilisations and sun/Climate and sun/Howell 2007 - A Preliminary note on Holocene climate.pdf
/media/bill/SWAPPER/Website - raw/Civilisations and sun/is in the separate file:Howell - Solar insolation for civilisations.pdf
/media/bill/SWAPPER/Website - raw/Civilisations and sun/Pandemics, health, and the Sun/_Pandemics, health, and the sun.html
/media/bill/SWAPPER/Website - raw/Climate and sun/Civilisations and sun/Howell - Mega-Life, Mega-Death and the Sun II, towards a quasi-predictive model of the rise and fall of civilisations.pdf
+-----+

Fix backtracking code, and see if older problems re-occur :
>> not much difference?  I wonder what's wrong?

First I must get rid of absolute links to my website (eg http://www.BillHowell.ca ...)
$ grep  'http://www.BillHowell.ca'  '/media/bill/ramdisk/html urls list.txt'  >'/media/bill/ramdisk/0_absolute links to website.txt'
>> There are 73 examples, most of which are in the "Conference Guides"
All but one supposedly fixed

I also redid all steps of coding as a check

OK - but the problem now is to simplify checks by making sure that ALL [<A HREF]", SRC="] links : 
1.	get rid of %20  
	(careful - mailtos need it!)
2. are relative addressing, so get rid of http://www.BillHowell.ca style links 
	(careful - must apply proper d_base truncations!!  This is manual)


08********08
#] 02Oct2020 website update - check all links
link  d_Qndfs 'website urls.ndf' -  test all links of a website, list faulty links

Yikes!! porn?
http://beastiegals.com/544170559.html
I put all old bookmarks in Website[-raw,] z_Old, and deleted those online

Also - banking!!!

04Oct2020 OK, now the program basically works with [internal, external, menu, pagePosn] links
- see summary table below
I suspect that even though >= 2/3 of links are OK, many of the 'fails' are actually OK
Next step is to track down examples and fix code.
ignore menues for now

+-----+
"$d_SWAPPER""/media/bill/SWAPPER/Website - raw/5_website linkType fname counts.txt" 
www.BillHowell.ca 03Oct2020 initial, update :  4Oct2020

Website stats for :  www.BillHowell.ca
Summary of the number of links by type [external, internal, menu, tableOfContent] and [OK, bad] : 

150 /media/bill/SWAPPER/Website - raw/6_extern urls fails.txt
238 /media/bill/SWAPPER/Website - raw/6_extern urls OK.txt
104 /media/bill/SWAPPER/Website - raw/7_intern urls fails.txt
267 /media/bill/SWAPPER/Website - raw/7_intern urls OK.txt
187 /media/bill/SWAPPER/Website - raw/8_menues urls fails.txt
112 /media/bill/SWAPPER/Website - raw/8_menues urls OK.txt
 48 /media/bill/SWAPPER/Website - raw/9_pgPosn urls fails.txt
269 /media/bill/SWAPPER/Website - raw/9_pgPosn urls OK.txt


08********08
#] 22Sep2020 rsync WebRaw -> Web

Dads paintings are links?!!! - check WebRaw
--dry-run dosesn't report changes t be made!!!???


08********08
#] 18May2020 Graphs of corona virus by country, grouped by region

Here's a quick chart on COVID-19 daily case rates in Anglophone countries (this chart is NOT the daily death rate!).  
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell%20-%20corona%20virus%20of%20countries,%20by%20region.html

Canada is : 
- doing really well compared to the [USA, UK, Australia], but far exceeds the effects in New Zealand
- at the lower end of the weekly fluctuations of Sweden (who did not push the [self-isolation, social distancing] as much, and didn't shut down their economy.
- doing poorly when compared to most of the world's population
It looks like European-descended countries are having a particularly hard time 

Charts for countries in most regions of the world, plus my crazy [questions, comments] can be found at : 
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell%20-%20corona%20virus%202020.html

Links to two spreadsheets for producing the charts are also provided on that web-page.  This makes it easier for the reader to produce their own charts, such as daily deaths.  


http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell%20-%20corona%20virus%202020.html

http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell%20-%20corona%20virus%20of%20countries,%20by%20region.html

http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Anglophone.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Northern%20Europe.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Southern%20Europe.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Scandanavia.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20North%20Africa.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Pacific%20Asia.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20South%20Asia.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Latin%20America.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Mesopotamia%20etc.png
http://www.BillHowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20daily%20Russia%20etc.png


http://www.BillHowell.ca/Pandemics, health, and the Sun/Howell - corona virus of countries, by region.html

Howell, OurWorldInData.org daily Anglophone.png
Howell, OurWorldInData.org daily Scandanavia.png
Howell, OurWorldInData.org daily Northern Europe.png
Howell, OurWorldInData.org daily Southern Europe.png
Howell, OurWorldInData.org daily North Africa.png
Howell, OurWorldInData.org daily Pacific Asia.png
Howell, OurWorldInData.org daily South Asia.png
Howell, OurWorldInData.org daily Latin America.png
Howell, OurWorldInData.org daily Mesopotamia etc.png
Howell, OurWorldInData.org daily Russia etc.png


/media/bill/SWAPPER/Website - raw/Pandemics, health, and the Sun/Howell - corona virus of countries, by region, collection.html



08********08
#] 15May2020 Scandanavia corona virus, response to Ian Hepher's Facebook post

/home/bill/PROJECTS/Investments/References/corona SARS2 virus
Howell, OurWorldInData.org Scandanavia daily-covid-deaths-per-million-3-day-avg.jpg
Howell, OurWorldInData.org Scandanavia daily-covid-cases-per-million-3day-avg.jpg
Howell, OurWorldInData.org Scandanavia daily-covid-[cases,deaths]-per-million-3day-avg.xcf


Yes, and this is quite interesting.  [Case, death] daily rates per million population are shown below.  
http://www.billhowell.ca/Pandemics,%20health,%20and%20the%20Sun/Howell,%20OurWorldInData.org%20Scandanavia%20daily-covid-[cases,deaths]-per-million-3day-avg.png

Amongst leading caucasian nations, Sweden is not exceptional at all - the real question is why are the other Scandanavian countries so much lower, even far lower than Canada?   I could always blurb out many ideas, but none of them seem to be very convincing right now.  They remind me of [Far Eastern, Southern] Asian nations.

On a daily death basis-per-million, Sweden is much higher than the rest of Scandanavia.  

On a daily cases basis-per-million,  Sweden has much higher rates than [Norway, Finland], but it is really Iceland that stands out on a per million population basis.  Given their small population, one might expect Iceland to have exceptional [high, low] excursions - one bad apple in a small place is all it may take?  Again on a rate basis (per population), Sweden is well below the earlier high levels of [Spain, Italy], is currently just lower than [USA, UK, Russia], and is currently just above Canada. 

You can easily select different countries and produce your won graphs from the spreadsheets linked on my covid web-page :  
www-billhowell-ca/Pandemics, health, and the Sun/Howell - corona virus 2020.html


08********08
#] 15May2020 Cool email notifications, eg Russ Hall

Russ Hall
$ find  "/media/bill/SWAPPER/Website/Cool emails"  -maxdepth 1  -name "*"  |   tr  \\n  \\0  |  xargs -0  -IFILE  grep  --with-filename  --line-number  -s  "Russ Hall"   "FILE"
+-----+
<b>To: 		</b>"Russ Hall. BPit Senior Executive coaching-mentoring-leadership. Calgary"<br>
            <td>Russ Hall. Senior Executive
              <td>Russ Hall <a class="moz-txt-link-rfc2396E"
<b>To: 		</b>"Russ Hall. BPit Senior Executive coaching-mentoring-leadership. Calgary"<br>
            <td>Russ Hall <a class="moz-txt-link-rfc2396E" href="mailto:">&lt;&gt;</a></td>
+-----+

$ find  "/media/bill/SWAPPER/Website/Cool emails"  -maxdepth 1  -name "*"  |   tr  \\n  \\0  |  xargs -0  -IFILE  grep  --with-filename  --line-number  -s  "Russ Hall"   "FILE"
+-----+
/media/bill/SWAPPER/Website/Cool emails/190505 Subject: RE: Neil Howell's birthday - Climate change summaries.html:11:<b>To: 		</b>"Russ Hall. BPit Senior Executive coaching-mentoring-leadership. Calgary"<br>
/media/bill/SWAPPER/Website/Cool emails/190505 Subject: RE: Neil Howell's birthday - Climate change summaries.html:184:            <td>Russ Hall. Senior Executive
/media/bill/SWAPPER/Website/Cool emails/190505 Subject: RE: Neil Howell's birthday - Climate change summaries.html:277:              <td>Russ Hall <a class="moz-txt-link-rfc2396E"
/media/bill/SWAPPER/Website/Cool emails/190510 Subject: =?UTF-8?Q?RE=3a_Top_Scientist_Resigns=3a_=27Global_Warming_is_a_=24?=.html:11:<b>To: 		</b>"Russ Hall. BPit Senior Executive coaching-mentoring-leadership. Calgary"<br>
/media/bill/SWAPPER/Website/Cool emails/190510 Subject: =?UTF-8?Q?RE=3a_Top_Scientist_Resigns=3a_=27Global_Warming_is_a_=24?=.html:274:            <td>Russ Hall <a class="moz-txt-link-rfc2396E" href="mailto:">&lt;&gt;</a></td>
+-----+

$ find  "/media/bill/SWAPPER/Website/Cool emails"  -maxdepth 1  -name "*"  |   tr  \\n  \\0  |  xargs -0  -IFILE  grep  --with-filename  --line-number  -s  "Russ"   "FILE"
+-----+
too many ... [Russia, Russo, etc]
+-----+

Backup those two files in new "Website secure", then edit in "C

08********08
#] 24Apr2020 put in for updates of ONLY changed dw_base html files (NYET!)
# NYET - can't do this, as menus may have changed 
# put in for updates of ONLY changed dw_base html files : 
		% first check to see if the output html file exists in ds_base, or needs updating!  ;  
		IF		( 1  >  (-  (EACH file_dateTimeStamp  (link  dw_base  page)  (link  ds_base  page)))) 
		THEN  update_webPage page  ;  
		ENDIF  ;  

08********08
#] 23Apr2020 du  --all  --max-depth=2  /home/bill/PROJECTS/System_maintenance

08********08
#] 22Apr2020 update_webPage  IS OP  webPage
old version : 

update_webPage  IS OP  webPage  
{  LOCAL		dir  base  extension 
				fin  fot  i_title  line  p_insert  p_out  title  dw_webSite  ;  
	NONLOCAL	pList  dw_base  ds_base  titles  ;  
																%  ;  
	dir  base  extension  :=  pName_extract_dirBaseExtn  webPage  ;  
	dw_webSite  	:=  strFull_strFront_extract_strEnd  dir  dw_base  ;  
	p_out				:=  link  ds_base  dw_webSite    base  '.'  extension  ;  
	fin				:=  open  webPage  "r  ;  
	fot				:=  open  p_out	 "w  ;  
	not_in_insert  :=  l  ;  
	do_insert		:=  l  ;  
																%  ;  
	WHILE  		(~= ??eof  (line  :=  readfile  fin))  DO  
		IF			(in_string  ':&file-insert &:'  line)  THEN   writefile fot  line  ;  
			% insert lines from title file  ;  
					do_insert  :=  l  ;  
					not_in_insert  :=  o  ;  
			title		:=  extract_pname  line  ;  
			i_title  :=  find_Howell  title  titles  ;  
			IF		(isfault i_title)		THEN  
					not_in_insert  :=  l  ;  
					writefile  fot  (link  '<!--- update_webPage error, title not recognized : '  title     ' --->')  ;  
			ELSE  
					p_insert  :=  execute (i_title  pick  pList)  ;  
				IF (file_exists  '-f'    p_insert)  THEN     
					pName_insertIn_fHand  p_insert  fot  ;  
				ELSE  
					writefile  fot  (link  '<!--- update_webPage error, p_insert doesnt exist : '  p_insert  ' --->')  ;  
				ENDIF  ;  
			ENDIF  ;  
		ELSEIF	(in_string  ':@insert-ender@:'  line) THEN   writefile fot  line  ;  
					not_in_insert  :=  l  ;  
		ELSEIF	not_in_insert								  THEN   writefile fot  line  ;  
		ENDIF  ;  
	ENDWHILE  ;  
																%  ;  
	EACH close  fin  fot  ;  
}  


08********08
#] 22Apr2020 find  "$d_webRawe"  "file-insert" "*.html" files

$ find  "/media/bill/SWAPPER/Website - raw/"  -maxdepth 3  -name "*.html"  |  sort  -u  >"/media/bill/ramdisk/full.txt"
$ find  "/media/bill/SWAPPER/Website - raw/"  -maxdepth 3  -name "*.html"   |  tr  \\n  \\0  |  xargs -0  -IFILE  grep  --with-filename  --line-number  "file-insert" "FILE"  |  sed  's#:.*##'  |  sort  -u  >"/media/bill/ramdisk/short sort.txt"

# $ diff  "/media/bill/ramdisk/full sort.txt"  "/media/bill/ramdisk/short sort.txt"  
$ diff  "/media/bill/ramdisk/full sort.txt"  "/media/bill/ramdisk/short sort.txt"  
11,25d10
< /media/bill/SWAPPER/Website - raw/fin Copyright ending.html
< /media/bill/SWAPPER/Website - raw/fin Footer1.html
< /media/bill/SWAPPER/Website - raw/fin footer1 Neil Howell.html
< /media/bill/SWAPPER/Website - raw/fin footer1 Paul Vauhan.html
< /media/bill/SWAPPER/Website - raw/fin footer1 Steven Wickson.html
< /media/bill/SWAPPER/Website - raw/fin footer1 Steven Yaskell.html
< /media/bill/SWAPPER/Website - raw/fin Footer2.html
< /media/bill/SWAPPER/Website - raw/fin Footer.html
< /media/bill/SWAPPER/Website - raw/fin Head_one (another copy).html
< /media/bill/SWAPPER/Website - raw/fin Head_one (copy).html
< /media/bill/SWAPPER/Website - raw/fin Head_one.html
< /media/bill/SWAPPER/Website - raw/fin Head_two (another copy).html
< /media/bill/SWAPPER/Website - raw/fin Head_two (copy).html
< /media/bill/SWAPPER/Website - raw/fin Head_two.html
< /media/bill/SWAPPER/Website - raw/fin organisations.html
31,49d15
< /media/bill/SWAPPER/Website - raw/Menu1.html
< /media/bill/SWAPPER/Website - raw/Menu2.html
< /media/bill/SWAPPER/Website - raw/Menu blogs.html
< /media/bill/SWAPPER/Website - raw/Menu crazy themes and stories1.html
< /media/bill/SWAPPER/Website - raw/Menu crazy themes and stories2.html
< /media/bill/SWAPPER/Website - raw/Menu crazy themes and stories.html
< /media/bill/SWAPPER/Website - raw/Menu earth, sun, astro, history1.html
< /media/bill/SWAPPER/Website - raw/Menu earth, sun, astro, history.html
< /media/bill/SWAPPER/Website - raw/Menu hosted subsites1.html
< /media/bill/SWAPPER/Website - raw/Menu hosted subsites.html
< /media/bill/SWAPPER/Website - raw/Menu Howell videos2.html
< /media/bill/SWAPPER/Website - raw/Menu Howell videos.html
< /media/bill/SWAPPER/Website - raw/Menu.html
< /media/bill/SWAPPER/Website - raw/Menu neural nets.html
< /media/bill/SWAPPER/Website - raw/Menu professional and resume.html
< /media/bill/SWAPPER/Website - raw/Menu projects1.html
< /media/bill/SWAPPER/Website - raw/Menu projects2.html
< /media/bill/SWAPPER/Website - raw/Menu projects.html
< /media/bill/SWAPPER/Website - raw/Menu software programming.html
53d18
< /media/bill/SWAPPER/Website - raw/Neural nets/MindCode/Computational neuro-genetic modelling.html
66d30
< /media/bill/SWAPPER/Website - raw/Personal/181211 Van versus Semi collision/181211 Howell van Crash on Highway 10 East - DrumhellerOnline.com.html
>> OK - most should NOT have "file-insert", but these two should?
+-----+
< /media/bill/SWAPPER/Website - raw/Neural nets/MindCode/Computational neuro-genetic modelling.html
66d30
>> ??I don't understand why it doesn't show up
>> just leave it for now
< /media/bill/SWAPPER/Website - raw/Personal/181211 Van versus Semi collision/181211 Howell van Crash on Highway 10 East - DrumhellerOnline.com.html
>> ignore this for now
+-----+



# enddoc