"$d_Qndfs""webSite/2_website updates [summary, ToDos, status].txt" # www.BillHowell.ca 07Nov2020 initial, taken from 'Website updates.ndf' 'website updates notes.txt' # This web-page is related to whole webSite processes # For each operator, see the relevant .ndf file 07Nov2020 webSite_convertBodyLinks : webSite_doAll For [track, fix]ing problems, see "$d_Qndfs""webSite/1_webSite instructions.txt" >> makes it FAR easier! [bash, QNial] tools Upload webSite : 24************************24 08********08 = '[#!: ' (5 take line) - full-line executeEmbed [#=; backtrack ;=#] - can occur anywhere in a line !!linkError!! - specific to links, to help track flawed links in a webSite geany: search&rep(but DON'T replace!), regexpr : > worry about this later... 03Jun2021 Stop for now - leave NN corrections to next round in 3-6 months 03Jun2021 Why are there 806 linkErrors but only 58 failed targetURLs? That is ~12 linkErrors/failed targetURLs might be Conference Guide menus? 04Jun2021 path_exists has ["r, "fw, "d] now. this is going to screw up a lot of legacy coding!!!! 04Jun2021 need to remove 'Climate and sun' redundant subDirs! (and probably others as well) 05Jun2021 ??circular updates [webRawe, webURLs_extract] ; 06Jun2021 can't get this to work! find grep "path_exists" to "path_exists outside of Qndfs.txt" 07Jun2021 urls_check - removed NONLOCALs (these are defined in header) Question - these don't have to be included? as QNial will look NONLOCAL? maybe better if I do declare as NONLOCAL? wait and see... 08Jun2021 webSite_link_counts -> writefile of 'Count of files on this webSite' not working see strings.ndf 09Jun2021 lftpUpload - rather than commenting out" code, it would be far safer to cl with arguments! 09Jun2021 Icebreaker needs a captured video of Part 1 of 6 08********08 01Jun2021 Overview of other situations to cluster cases and tackle together +--+ Conference Guide - is it updated or excluded? >> leave this for last... >> these are included and updated, but they don't have embedded menu etc inserts /media/bill/WebSite/!!linkError!!WCCI2020 mass email [SS,Comp,Tut,Wrkshp] 191025 Howell.html /media/bill/Dell2/Website - raw/Neural nets/Conference guides/Publications website/CrossCheck - Publications Chair explanation of CrossCheck results and analysis.html >> exists in PubGuide /media/bill/WebSite/!!linkError!!Neural nets/Conference guides/Author guide website/N-19557 wrong paper [size, margin]s.pdf /media/bill/WebSite/!!linkError!!webWork files/confMenu_authors.html >> not present - is it somewhere else? backup drive, but confMenus aren't being used yet files shouldn't be linked! /media/bill/WebSite/!!linkError!!IJCNN2019 mass email 180925 v180925 Howell final.html /media/bill/WebSite/!!linkError!!IJCNN2019 mass email 181102 v181102 Howell with Plenaries, MDPI Genisama logos.html /media/bill/WebSite/!!linkError!!IJCNN2019 mass email 190108 v190102 Howell paper deadline, Plenary Speakers.html 08********08 27Oct2020 Status submenus : Home n/a Neural Nets All work except [Neural Nets] Projects most work still, but NOT [Puetz, Randall] Software programming & code all work Professional & Resume all work Publications & reports n/a Howell-produced videos all work Blogs all work Cool stuff n/a - just lists directory content Crazy themes and stories all work Hosted sub-sites all work Neil Howell's Art all work # commercial webSite management systems (Joomla, Modx, Mambo and other quality Content Management System) # 25Oct2020 Corrections cycle in d_webRaw # To fix previous conversions of web-[pages, site], it's much easier just to : # 1. dirBackup_restoreTo_paths webpages from an earlier date # 2. add corrections to webPage_convertEncoding # 3. webSite_convert # Then update whole webSite - one-way flow of html files, so backups are not an issue : # 4. webSite_update 08********08 Analysis of webSite links for most recent results, see : "$d_webWork"'webSite summary of [fail, unknown, OK,total] links.txt' /media/bill/Dell2/Website - raw/webWork files/webSite summary of [file, link] cart [types, counts].txt 8Jun2021webSite stats for : www.BillHowell.ca Summary of the number of [webPages, targets, links] by [type, count] : +-----+ Count of files on this webSite (09Jun2021 - this does currently work after changes) +---------------------------------+ ||pathDir[pass,sortBy] |all |html|| +---------------------------------+ ||PathsSortedByFname |6238|6238|| +---------------------------------+ ||SubDirsSortedByFname |6238|7300|| +---------------------------------+ ||FnamesSortedByFname |383 |206 || +---------------------------------+ ||PathsSortedByPath |206 |206 || +---------------------------------+ ||SubDirsSortedBySubdir|222 |59 || +---------------------------------+ +-----+ Count of links on this website : +----------------------------------------------------------+ ||8799|all linkLines in webPages of this webSite (1) || +----------------------------------------------------------+ ||107 |all linkErrors on the webSite (2) || +----------------------------------------------------------+ ||63 |all linkErrorNotOnlys - with other text in link (3)|| +----------------------------------------------------------+ ||44 |all linkErrorsOnlys - no other text in link (4)|| +----------------------------------------------------------+ +-----+ Counts below are the number of unique TARGETED [file, dir]s (URLs) of links Failures : There may be some overlap between "errors" and the other 3 failure types +-----------------+ ||26|errors list || +-----------------+ ||30|extern fails|| +-----------------+ ||1 |intern fails|| +-----------------+ ||0 |bkmkEx fails|| +-----------------+ Unknowns - links not directly tested by urls_check tool (6) (I havent written code to really show [OK, fail]) +-----------------+ ||75 |mailto list|| +-----------------+ ||309|bkmkIn list|| +-----------------+ OKs - these links have been shown to work : +---------------+ ||275|extern OK|| +---------------+ ||498|intern OK|| +---------------+ ||26 |bkmkEx OK|| +---------------+ Summary of targetURL counts : +-------------------------+ || 57|failed targetURLs|| +-------------------------+ || 384|unknown targetURLs|| +-------------------------+ || 799|OK targetURLs|| +-------------------------+ ||1240|total || +-------------------------+ +-----+ Footnotes : (1) "linkErrors" arise from internal-to-this-webSite links which could not be auto-reconstructed many of the problems result from 2 or 3 major restngs of my [computer, directory]s some are from typos etc ~8 links per target file on average. Menus contain the VAST majority of all links Links do NOT include those in [adobe pdf, word processing, spreadheet, etc] files I use Linux, with LibreOffice programs plus occasionally Open Latex. I do have [Apple, Microsoft] from other people or from my past work (2) Many linkErrors are not really a problem : In this case, empty links are a good thing for me, albeit perhaps annoying to the reader. (3) This is a key list of items to work on (4) As of 03Jun2021, ~40 or 5% of linkErrors were "empty links", which were put in my webPages as a reminder that [thought, list]s are incomplete. (5) "Failures" result from actual checks of the urls "curl -Is" - was used to check external online urls unix [ -d path] and [ -f path ] were used to check this website (on hard drive prior to upload) (6) Unknowns - are not checked (yet) by my software tools : mailtos are [old, deactivated for privacy], they wont work now bkmkIn - these bookmarks are internal to the same webPage that they appear in. Users may easily find the intended location in same webPage Obviously, there is a tight relation between "linkErrors", and targeted [url, subDir, fname] "failures". But they are not the same, as there are often multiple links to the same target, and the "Unknowns" arent tested 08********08 Old ToDos 27Oct2020 rsync script for PROJECTS to SWAPPER updates (eg "Software programming" etc) rsync-equivalent for webSite online update 27Oct2020 see list of a few web-pages to fix, plus the "Status" above 27Oct2020 I need to check for [old, dead] website material and delete it Best of use "delete if not in d_webSite update eg rsync or whatever I can use for the webite update besides FileZilla if it can't do it. 27Oct2020 Home page : series of photos of me over time... last photo on right - me now, post van crash 27Oct2020 http://www.billhowell.ca/page Software programming.html - add 'system maintenance' description - Website [update, url verification]s descriptions & links - rsync backups [QNial (old), bash] - Hybridized [QNial, Unic command, bash] environment - challenge of [quote, apo]s 27Oct2020 Add "embedded executable" description to [software, Qnial] projects kind of a [cheap, crappy] complement to javascript at the web-page generation stage, not during use. I should use javascript? - RELIABLE html or other text file section numbering as example 27Oct2020 QNial web-page not linked in menu 31Oct2020 I need a '[#=; webPage_buildTableOfContents ;=#]' [operator, embedded executable] 06Nov2020 if not an fname - check for a legitimate subDir!! Then I don't have to change it. 07Nov2020 path - need more than just last subDir! 07Nov2020 webSite_convertBodyLinks : 22Nov2020 Amazon links are un-reliable 13Dec2020 extra
between last menu and 'normalStatus.html' for some webPages 13Dec2020 cannot have `& ? 13Dec2020 other unsolved problems 13Dec2020 check for duplicate fnames of webPages!!! 14Dec2020 apo (apostrophe) problems : !!linkError!!Allegre's second thoughts.pdf !!linkError!!Bill Howells videos/120214 Venus et Mars, au dela d'une histoire d amour/Mythology.flv !!linkError!!Climate - Kyoto Premise fraud/_Kyoto Premise - the scientists aren't wearing any clothes.html !!linkError!!Lucas's Universal Force for electrodynamics, gravity, mass, etc !!linkError!!Paul L Vaughan/Vaughan 120324 The Solar Cycle's Footprint on Terrestrial Climate.PDF !!linkError!!Software programming & code/bin/bin - Howell's web-page.html 26May2021 should References be excluded from d_webSite? for now, yes? 26May2021 (from earlieer note, not dated) curl sites : IF (OR ('200' '300' '301' '302' EACHLEFT in_string curlHdr)) I need to make an easy list of code explanations [OK, fail] 30May2021 Problem with `& i[subDir,fname]s : /media/bill/WebSite/!!linkError!!Bill Howells videos/140214 Nazis saved Norwegians video/Nazis saved Norwegian lives.flv/media/bill/WebSite/!!linkError!!Charvatova solar inertial motion & activity/Verification/ /media/bill/WebSite/!!linkError!!Software programming & code/bin/SSH/ /media/bill/WebSite/!!linkError!!Software programming & code/System_maintenance/ /media/bill/WebSite/!!linkError!!Solar modeling and forecasting/_Solar modeling & forecasting.html 25May2021 problem with updating d_webSite menues, persisten linkErrors! 25May2021 backup of html files not working 28May2021 Test webPageRawe_update, current issue is linkErrors 26May2021 DO NOT do this in the header - may not need : IF (NOT path_exists d_htmlBackup) THEN host link 'mkdir "' d_htmlBackup '" ' ; ENDIF ; 26May2021 d_htmlBackup is also defined in : webAllRawOrSite_update required in webPage[Rawe,Site]_update, so above Note : webSite_convert creates a new backup directory every time it is used, eg % create a new backup directory for every use of webSite_convert, as damage can be VERY time-costly ; d_htmlBackup := link d_webRawe 'z_Archive/' timestamp_YYMMDD_HMS ' backups/' ; host link 'mkdir "' d_htmlBackup '" ' ; Therefore, there is a conflict between the two, which will see-saw between them >> there is a problem here!!! 02Jun2021 '& problem in [grep, sed] sed special characters, see "$d_SysMaint""Linux/sed notes.txt" in the list of characters matched by a bracket expression '"$.*/[\]^ In the replacement text: '&\/\n' # enddoc