********************* loaddefs link d_Qndfs 'file_ops.ndf' Predefine local ops to avoid sequencing problems with initial qnial loaddefs +-----+ [exist, size, dateTimeStamp, extract, change] names of [pth, dir] [in, is] normally [o,l] (but can be faultL), check [faultL, null] pthNm_extract_fname IS OP pth - extract the fileName from a path pthNm_extract_dir IS OP pth - extract the directory from a path pthNm_extract_dirFname IS OP pth - extract the [dir, fname] from a path pthNm_extract_dirBaseExtn IS OP pth - extract [dir, base, extension] from a path-file name pthNm_extract_subDirFname IS OP pth dirBase - returns (subDir fname) from a path pinn_dateTimeStamp IS OP pth - return the dateTimeStamp of the file pinn_change_extension IS OP pinn fileExtension_new - new fileExtension pthname, no change to pinn povr_delete IS OP pth - delete a file fname_in_fnameGlobalList IS OP fname - test if fname appears in a global list of fnames See 'webSite maintain [menu, header, footer, body] links, TableOfContents.ndf' easy to adapt to other programs? included as a reminder subDir_in_subDirGlobalList IS OP subDir - find the longest valid subSubDir in subDir, within a global-defined root dir See 'webSite maintain [menu, header, footer, body] links, TableOfContents.ndf' easy to adapt to other programs? included as a reminder +-----+ exists of [dir, pth] : pth_typ_exists IS OP pth typ - returns OR[true, fault], wrt to [pinn, padd, pnew, povr, dold, dnew] pthOrDir_size IS OP pth - returns [dir, pth]size in [T,G,M,K]-byte format, diSize is recursive total pinn_isEmpty IS OP pth - returns l if true and o if false, applicable to [dir, pth]s Warning : if a path does not exist, then as per Linux convention, this returns o (non-existent files aren't empty!) I had bandone +-----+ checks for [pth, dir]s : checkCanWrite_pth_pthTyp IS OP pth pthTyp - check validity of proposed writefile pthL_faultLL_shortenTo_Codes IS OP pthL faultLL - for _tests, maybe general use [pthL, faultL] for faults are converted to codes for brevity in the output of course, a resultCode of null (true) is kept the same +-----+ [backup, restore] cart [pth, dir]s pinn_backupDatedTo_zArchive IS OP pinn - SAFE WORKING backup file to a z_Archive dated today (create?) zArchive is [specific, with respect] to each path, it is not an overall zArchive Notice that there is no error output (write) - that must be handeled by the calling optr! pinn_find_latestZarchive IS OP pinn - look for the most recent version of fname in d_zArchive pZarchive_restore IS OP povr - restore most recent pinn_backupDatedTo_zArchive of a file, retain the backup pinnL_backupTo_dirFlat IS OP pinnL d_backup - backup multi-dir paths to a (created) FLAT dir Note that these backups are NOT dated There will be a problem with fnames that are the same!!! This is often used for d_webRawe work, placed into a dated directory for a [pinnL, dir] operation dirFlat_restoreTo_povrL IS OP d_backup povrL - restore dated paths from a FLAT d_backup, created with pinnL_backupTo_dirFlat +-----+ [find, count, in] [chr, str, word]s in d_Qndfs [file,dir]s - including [find,grep] simple command line stuff d_Qndfs [bash script, QNial [optr,var,etc]]s - [pth, lineNum]s (Linux commands) for now I just have [find,grep] simple command line stuff paren_unbalanced - see pinn_chr_writeScreenUnbalancedLines pinn_chr_writeScreenUnbalancedLines IS OP pinn str - write lines with unbalanced chr str_in_pinn IS OP str pinn - boolean indicator that str is in file pinn just uses Unix grep in host mode!! pinn_lineNumOf_chrSet IS OP pinn chrSet - find [in, NOT in] characters in path pinn_searchL_options_grepTo_povr IS OP pinn searchL options povr - povr = null for stdOut, assumes that special grepSearchChrL have been escaped grep options : '' (null) for normal grep, ['-w', ?'-E '\w+'] searches for words, '-c' for count -o only prints the portion of the line that matches +-----+ replace [chr, word, str] in [file, directory] pinn_removeWhiteLines_pnew - use grep (example overwrites original path) : eg host link 'grep "\S" "' p_temp1 '" >"' p_temp2 '" && mv "' p_temp1 '" "' p_temp1 '"' ; pinn_removeWhiteLines_pnew IS OP pinn pnew - required to use emails as attachments searchReplaceL_options_make_sedExpr IS OP searchReplaceL options - create a sedExpr for searchReplaceL option = ['\b' for words, null for general string relacement (DANGEROUS!)] povrL_searchReplaceL_options_sed IS OP povrL searchReplaceL options - replace [str, word]L in each povrL, can use null for stdout option = ['\b' for words, null for general string relacement (DANGEROUS!)] povrL_searchReplaceL_options_sedMaxBatches IS OP povrL searchReplaceL options - uses povrL_searchReplaceL_options_sed, but does searchReplaceL in batches so as to not swamp sed option = ['\b' for words, null for general string relacement (DANGEROUS!)] povr_searchReplaceL_nonsedReplace IS OP povr searchReplaceL - [pinn lineOldL] to [pnew, lineNewL] AVOIDs sed, therefore NOT affected by sed restricted : search $.*/[\]^ replace &\/\n default backup by pinn_backupDatedTo_zArchive, overwrites original path dir_pExtn_options_searchReplaceL_sedReplace IS OP dir pExtn options searchReplaceL pExtn allows files of a specific exten to be targeted, wordOld != wordNew, unique by default, this is recursive in dir! +-----+ [read, write] cart [str, ary, listOfStr, listOfStrPair, UnixCmd][,L] pinn_read_strL IS OP pinn - simple read of file as a list (1D array) of strings, one per line pinn_readExecuteLines_aryL IS OP pinn - make a list of the results of executing each line of pinn "apo-enclose" strings unless they are [variable, operator, transformer, etc] on each line pinn_removeWhiteLines_pnew IS OP strL pth pthTyp - [append, over-write] a list of strings to pth makes use of global variable chrMaxPerLine, pthTyp = ["pinn "pnew "povr], (not for dirs) strL_write_fout IS OP strL fout - write a list of strs to a single output line apo-enclosed separated by spaces, output line to fout (not a path!) WARNING: assumes length(strL) < max strLength for fileInput strPL_write_fout IS OP strPL fout - write a list of strPairs (strPL) as one strPair per line str_write_povr IS OP str povr - replace povr with str, then use str_write_padd (below) str_write_padd IS OP str pthOrScreen - append one str at a time stdout: padd = null, as stdout is automatically append mode strL_write_povr IS OP strL povr - replace povr (if exists) with strL ary_writePictureTo_pout_pthTyp IS OP ary pout pthTyp - [append, over-write] a list of strings to pout makes use of global variable chrMaxPerLine, pthTyp = ["pinn "pnew "povr], (not for dirs) +-----+ [sort, catenate] file pinn_sortupUnixTo_pnew IS OP pinn pnew - sort text file, by default pnew cannot be over-written Warning : Unix sort is different from QNial sort results... pinn_sortupQNialTo_pnew IS OP pinn pnew pinnL_sortCatTo_pnew IS OP pinnL pnew - sort each file in pinnL, concatenate to pnew +-----+ file sections [add, remove, whatever] pinn_cutHeadTail_pnew IS OP headLine tailLine pinn pnew - ignores [< headLine, > tailLine] pinn_yahooNewsItemsExtract_pnew IS OP pinn pnew - res ipsa loquitor see link d_Qndfs 'economics, markets/winURL yahoo finance news download.ndf' +-----+ file handles and embedded executables - eg to insert [menu, header, footer]s into html files : Although built for webSite maintenance, with adaptation these may have far more general applicability. see 'Website updates.ndf' for : internalLinks_return_backupSubDirFnames - for an an internal link, where SubDir goes into d_webSite internalLinks_return_relativepth - also using executeEmbeds when present strEmbed_execute_strOut - execute embeds in line as an example : webPage_update - update pinn to d_out using executeEmbeds pinn_insertIn_fHand IS OP p_insert fHand - insert p_insert into fHand at the current writefile point pWebWorkL_change_headFoot IS OP povrL - change all whole-line-embeds for povrL this is currently d_webRaw specific, used to build Howell's webSite [menu, header, footer, etc]s strEmbed_execute_strOut IS OP str phraseValueList - execute embeds in line, return a str '[#!: ' is for full-line embeds; '[#=; ' ' ;=#]' are for embeds within a line normally, this would be in strings.ndf, but more clear when plahere with related code create_pEmbedNmTempUniq IS - generate a unique p_temp fname pinn_phrValueL_executeEmbeds IS OP pinn phraseValueList - execute embeds in path, [output to, return] p_temp '[#!: ' is for full-line embeds; '[#=; ' ' ;=#]' are for embeds within a line pinn_executeEmbedsInsertIn_fHand IS OP pinn phraseValueList - execute embeds, insert in fileHandle +-----+ URL [down, up]-load url_downloadTo_pnew IS OP URLname pnew - command line download url_downloadTxtTo_povr IS OP url winTitle povr - download text only (no [html, css] code) +-----+ [read, write] special [table, Unix cmdResult] formats pinn_read_csvTable IS OP pinn - read a csv table from file, cells to [apo-str, numeric] tbl_writeCSVformatTo_pnew IS OP table pnew - writefile table to path, [tab-separated cols, apo-text] important - assumes 2D table of ONLY [string, atomic data] types (eg not lists of numbers...) pHeaderTitleTabTbl_quoteTitles IS OP povr - quote titles, as per gnuplot header is at top of file, each line starts with '# ' str_removeRepeat_chr IS OP str chrr - temporary optr until ary_subP_replace_aryOut is working pTitleOneLineTabTbl_quoteTitles IS OP povr - quote titles, as per gnuplot header is at top of ile, each line starts with '# ' pinn_readExecute_tabTbl IS OP pinn - read tbl of tab delineated cols convert "cells" to proper type, eg spreadsheet of : - [str, phr, int, real, bool]s, NO chr_tabs within strings, just to separate cols ignores 'comment lines' that start with `#, must have NO empty lines! pinn_readStr_tabTbl IS OP pinn - read file as a tbl with tab delineated cols eg spreadsheet, ignores lines that start with `#, must have NO empty lines! all itms are strings, NO chr_tabs within strings, just to separate cols tblWhole_writeTabSep_padd IS OP tblWhole padd - append tab-separated "whole table" to pth tblWhole_writeTabSep_povr IS OP tblWhole povr - overwrite mode handles [chr, phr, bol, str] so far pinn_read_wcUnix IS OP pinn - read file of Unix "wc" command outputs ONLY! read file as a table with tab delineated cols (eg txt from spreadsheet) line count of file : tonumber first (host_result link 'wc -l "' '" | sed '' winID_copyAllTextTo_pnew IS OP winID pname - copy text of webPage and pre-pend to file see 'windows [open, close, ID, title], [get,set,move] cart [posn,size].ndf' 20Oct2021 I have lost this!!??!! +-----+ povr_addISOPdebug[, _marked] - add debug lines prior to optr descriptions to track loaddefs of path povr_addISOPdebug IS OP povr - add ['IF debug', '#] '] to optr descriptions to help debugging via loaddefs this is for ndf files whose optrs do NOT have a menu TableOfContents marker (line starts with '#] ') povr_addISOPdebug_marked IS OP povr - add only 'IF debug' to optr descriptions already having '#] ' +-----+ [debug, log] file optrs - 21Oct2021 not used much any more... write_debug IS OP AAA - writes debug information to the screen writefile_debug IS OP AAA - killfile_debug IS - writefile_log IS OP AAA - logs picture of array to d_QNial_temp '0_QNial log.txt' killfile_log IS - erases logfile d_QNial_temp '0_QNial log.txt'