"$d_Qndfs""webSite/2_website updates [summary, ToDos, status].txt"
# www.BillHowell.ca 07Nov2020 initial, taken from 'Website updates.ndf' 'website updates notes.txt'
# This web-page is related to whole webSite processes
# For each operator, see the relevant .ndf file
07Nov2020 webSite_convertBodyLinks : webSite_doAll
For [track, fix]ing problems, see "$d_Qndfs""webSite/1_webSite instructions.txt"
>> makes it FAR easier! [bash, QNial] tools
Upload webSite :
24************************24
08********08
= '[#!: ' (5 take line) - full-line executeEmbed
[#=; backtrack ;=#] - can occur anywhere in a line
!!linkError!! - specific to links, to help track flawed links in a webSite
geany: search&rep(but DON'T replace!), regexpr : > worry about this later...
03Jun2021 Stop for now - leave NN corrections to next round in 3-6 months
03Jun2021 Why are there 806 linkErrors
but only 58 failed targetURLs?
That is ~12 linkErrors/failed targetURLs
might be Conference Guide menus?
04Jun2021 path_exists has ["r, "fw, "d] now. this is going to screw up a lot of legacy coding!!!!
04Jun2021 need to remove 'Climate and sun' redundant subDirs! (and probably others as well)
05Jun2021 ??circular updates [webRawe, webURLs_extract] ;
06Jun2021 can't get this to work! find grep "path_exists" to "path_exists outside of Qndfs.txt"
07Jun2021 urls_check - removed NONLOCALs (these are defined in header)
Question - these don't have to be included? as QNial will look NONLOCAL?
maybe better if I do declare as NONLOCAL?
wait and see...
08Jun2021 webSite_link_counts -> writefile of 'Count of files on this webSite' not working
see strings.ndf
09Jun2021 lftpUpload - rather than commenting out" code, it would be far safer to cl with arguments!
09Jun2021 Icebreaker needs a captured video of Part 1 of 6
08********08
01Jun2021 Overview of other situations to cluster cases and tackle together
+--+
Conference Guide - is it updated or excluded?
>> leave this for last...
>> these are included and updated, but they don't have embedded menu etc inserts
/media/bill/WebSite/!!linkError!!WCCI2020 mass email [SS,Comp,Tut,Wrkshp] 191025 Howell.html
/media/bill/Dell2/Website - raw/Neural nets/Conference guides/Publications website/CrossCheck - Publications Chair explanation of CrossCheck results and analysis.html
>> exists in PubGuide
/media/bill/WebSite/!!linkError!!Neural nets/Conference guides/Author guide website/N-19557 wrong paper [size, margin]s.pdf
/media/bill/WebSite/!!linkError!!webWork files/confMenu_authors.html
>> not present - is it somewhere else? backup drive, but confMenus aren't being used yet
files shouldn't be linked!
/media/bill/WebSite/!!linkError!!IJCNN2019 mass email 180925 v180925 Howell final.html
/media/bill/WebSite/!!linkError!!IJCNN2019 mass email 181102 v181102 Howell with Plenaries, MDPI Genisama logos.html
/media/bill/WebSite/!!linkError!!IJCNN2019 mass email 190108 v190102 Howell paper deadline, Plenary Speakers.html
08********08
27Oct2020 Status submenus :
Home n/a
Neural Nets All work except [Neural Nets]
Projects most work still, but NOT [Puetz, Randall]
Software programming & code all work
Professional & Resume all work
Publications & reports n/a
Howell-produced videos all work
Blogs all work
Cool stuff n/a - just lists directory content
Crazy themes and stories all work
Hosted sub-sites all work
Neil Howell's Art all work
# commercial webSite management systems
(Joomla, Modx, Mambo and other quality Content Management System)
# 25Oct2020 Corrections cycle in d_webRaw
# To fix previous conversions of web-[pages, site], it's much easier just to :
# 1. dirBackup_restoreTo_paths webpages from an earlier date
# 2. add corrections to webPage_convertEncoding
# 3. webSite_convert
# Then update whole webSite - one-way flow of html files, so backups are not an issue :
# 4. webSite_update
08********08
Analysis of webSite links
for most recent results, see : "$d_webWork"'webSite summary of [fail, unknown, OK,total] links.txt'
/media/bill/Dell2/Website - raw/webWork files/webSite summary of [file, link] cart [types, counts].txt
8Jun2021webSite stats for : www.BillHowell.ca
Summary of the number of [webPages, targets, links] by [type, count] :
+-----+
Count of files on this webSite (09Jun2021 - this does currently work after changes)
+---------------------------------+
||pathDir[pass,sortBy] |all |html||
+---------------------------------+
||PathsSortedByFname |6238|6238||
+---------------------------------+
||SubDirsSortedByFname |6238|7300||
+---------------------------------+
||FnamesSortedByFname |383 |206 ||
+---------------------------------+
||PathsSortedByPath |206 |206 ||
+---------------------------------+
||SubDirsSortedBySubdir|222 |59 ||
+---------------------------------+
+-----+
Count of links on this website :
+----------------------------------------------------------+
||8799|all linkLines in webPages of this webSite (1) ||
+----------------------------------------------------------+
||107 |all linkErrors on the webSite (2) ||
+----------------------------------------------------------+
||63 |all linkErrorNotOnlys - with other text in link (3)||
+----------------------------------------------------------+
||44 |all linkErrorsOnlys - no other text in link (4)||
+----------------------------------------------------------+
+-----+
Counts below are the number of unique TARGETED [file, dir]s (URLs) of links
Failures :
There may be some overlap between "errors" and the other 3 failure types
+-----------------+
||26|errors list ||
+-----------------+
||30|extern fails||
+-----------------+
||1 |intern fails||
+-----------------+
||0 |bkmkEx fails||
+-----------------+
Unknowns - links not directly tested by urls_check tool (6)
(I havent written code to really show [OK, fail])
+-----------------+
||75 |mailto list||
+-----------------+
||309|bkmkIn list||
+-----------------+
OKs - these links have been shown to work :
+---------------+
||275|extern OK||
+---------------+
||498|intern OK||
+---------------+
||26 |bkmkEx OK||
+---------------+
Summary of targetURL counts :
+-------------------------+
|| 57|failed targetURLs||
+-------------------------+
|| 384|unknown targetURLs||
+-------------------------+
|| 799|OK targetURLs||
+-------------------------+
||1240|total ||
+-------------------------+
+-----+
Footnotes :
(1) "linkErrors" arise from internal-to-this-webSite links which could not be auto-reconstructed
many of the problems result from 2 or 3 major restngs of my [computer, directory]s
some are from typos etc
~8 links per target file on average. Menus contain the VAST majority of all links
Links do NOT include those in [adobe pdf, word processing, spreadheet, etc] files
I use Linux, with LibreOffice programs plus occasionally Open Latex.
I do have [Apple, Microsoft] from other people or from my past work
(2) Many linkErrors are not really a problem :
In this case, empty links are a good thing for me, albeit perhaps annoying to the reader.
(3) This is a key list of items to work on
(4) As of 03Jun2021, ~40 or 5% of linkErrors were "empty links", which were put
in my webPages as a reminder that [thought, list]s are incomplete.
(5) "Failures" result from actual checks of the urls
"curl -Is" - was used to check external online urls
unix [ -d path] and [ -f path ] were used to check this website
(on hard drive prior to upload)
(6) Unknowns - are not checked (yet) by my software tools :
mailtos are [old, deactivated for privacy], they wont work now
bkmkIn - these bookmarks are internal to the same webPage that they appear in.
Users may easily find the intended location in same webPage
Obviously, there is a tight relation between "linkErrors",
and targeted [url, subDir, fname] "failures". But they are not the same,
as there are often multiple links to the same target, and the "Unknowns" arent tested
08********08
Old ToDos
27Oct2020 rsync script for PROJECTS to SWAPPER updates (eg "Software programming" etc)
rsync-equivalent for webSite online update
27Oct2020 see list of a few web-pages to fix, plus the "Status" above
27Oct2020 I need to check for [old, dead] website material and delete it
Best of use "delete if not in d_webSite update eg rsync or whatever I can use for the webite update besides FileZilla if it can't do it.
27Oct2020 Home page : series of photos of me over time... last photo on right - me now, post van crash
27Oct2020 http://www.billhowell.ca/page Software programming.html
- add 'system maintenance' description
- Website [update, url verification]s descriptions & links
- rsync backups [QNial (old), bash]
- Hybridized [QNial, Unic command, bash] environment - challenge of [quote, apo]s
27Oct2020 Add "embedded executable" description to [software, Qnial] projects
kind of a [cheap, crappy] complement to javascript at the web-page generation stage, not during use. I should use javascript?
- RELIABLE html or other text file section numbering as example
27Oct2020 QNial web-page not linked in menu
31Oct2020 I need a '[#=; webPage_buildTableOfContents ;=#]' [operator, embedded executable]
06Nov2020 if not an fname - check for a legitimate subDir!! Then I don't have to change it.
07Nov2020 path - need more than just last subDir!
07Nov2020 webSite_convertBodyLinks :
22Nov2020 Amazon links are un-reliable
13Dec2020 extra
between last menu and 'normalStatus.html' for some webPages
13Dec2020 cannot have `& ?
13Dec2020 other unsolved problems
13Dec2020 check for duplicate fnames of webPages!!!
14Dec2020 apo (apostrophe) problems :
!!linkError!!Allegre's second thoughts.pdf
!!linkError!!Bill Howells videos/120214 Venus et Mars, au dela d'une histoire d amour/Mythology.flv
!!linkError!!Climate - Kyoto Premise fraud/_Kyoto Premise - the scientists aren't wearing any clothes.html
!!linkError!!Lucas's Universal Force for electrodynamics, gravity, mass, etc
!!linkError!!Paul L Vaughan/Vaughan 120324 The Solar Cycle's Footprint on Terrestrial Climate.PDF
!!linkError!!Software programming & code/bin/bin - Howell's web-page.html
26May2021 should References be excluded from d_webSite? for now, yes?
26May2021 (from earlieer note, not dated)
curl sites : IF (OR ('200' '300' '301' '302' EACHLEFT in_string curlHdr))
I need to make an easy list of code explanations [OK, fail]
30May2021 Problem with `& i[subDir,fname]s :
/media/bill/WebSite/!!linkError!!Bill Howells videos/140214 Nazis saved Norwegians video/Nazis saved Norwegian lives.flv/media/bill/WebSite/!!linkError!!Charvatova solar inertial motion & activity/Verification/
/media/bill/WebSite/!!linkError!!Software programming & code/bin/SSH/
/media/bill/WebSite/!!linkError!!Software programming & code/System_maintenance/
/media/bill/WebSite/!!linkError!!Solar modeling and forecasting/_Solar modeling & forecasting.html
25May2021 problem with updating d_webSite menues, persisten linkErrors!
25May2021 backup of html files not working
28May2021 Test webPageRawe_update, current issue is linkErrors
26May2021 DO NOT do this in the header - may not need :
IF (NOT path_exists d_htmlBackup) THEN host link 'mkdir "' d_htmlBackup '" ' ; ENDIF ;
26May2021 d_htmlBackup is also defined in : webAllRawOrSite_update
required in webPage[Rawe,Site]_update, so above
Note : webSite_convert creates a new backup directory every time it is used, eg
% create a new backup directory for every use of webSite_convert, as damage can be VERY time-costly ;
d_htmlBackup := link d_webRawe 'z_Archive/' timestamp_YYMMDD_HMS ' backups/' ;
host link 'mkdir "' d_htmlBackup '" ' ;
Therefore, there is a conflict between the two, which will see-saw between them
>> there is a problem here!!!
02Jun2021 '& problem in [grep, sed]
sed special characters, see "$d_SysMaint""Linux/sed notes.txt"
in the list of characters matched by a bracket expression '"$.*/[\]^
In the replacement text: '&\/\n'
# enddoc