********************* "$d_fileops"'fileops.sh' - library of handy bash functions for file needs # do NOT run this directly!, use, for example, "$d_bin"'fileops run.sh' +-----+ Setup [func, process]*[Start, Ender] see : "$d_bin""standard header.sh" this is NOT used formain libraries [fileops, strings, etc].sh ([usefulness, efficiency] issues) +-----+ ToDos 19Nov2025 pinnL_pSedExpL_get_pStrL: use tac (reverse of cat) to reverse whole file! 02Jun2021 I have assigned not-so-random numbers to FILE with read loops to avoid "crossover effects of nesting" of this bash library. +-----+ fprintf #() LMDE doesn't have this!! not worth doing... +-----+ test function [tool, template]s - must see "$d_bin"'0_test/fileops/fileops test.sh' : funcNm_pOvrL_argL_execute #(funcNm pOvrL argL) flexArgs +-----+ [symbol, function] symbol codes - to prevent "cross-over" between library calls code_get #(no args) [symbol, function] codes: list all being used to check against new random codes code_make #(fractionSeed) new [symbol, function]s: generate [new, unique] code +-----+ [dir, pth] basic functions I do NOT yet have [p,d][new, ovr, add, etc] distinctions that I created in QNial! chmod_Howell #() change standard access rights permissions [d_web, online] : root - don't touch!!, dir permissions = 755, file = 644 chown_Howell #(d_target) change standard access rights permissions [d_web, online] : root - don't touch!!, dir permissions = 755, file = 644 pth_get_fnam #(pth) get fname with extension, pth must have at least one '/' pth_getExists_fnam #(pth) get fname with extension, pth must have at least one '/' pth_get_fNamNoExt #() get fname without extension pth_get_ext #() pth must exist or gives error fNam_get_fNamNoExt #(fNam) get fNam without extension (NOT pth!) fNam_get_ext #(fNam) retrieve fName extension (NOT pth!) pth_get_dir #(pth) retrieve dir holding pth, dir must exist or error dInn_ctime_get_pCtimeL_cp_dCtime #() tMods find pPthL, ctime in days, cp to dCtime pth_get_dArchive #(pInn) returns dArc for pInn ensures that z_Archive is created if it doesn't exist, pth_get_pArcDate #(pth) of form "$dArc$fNoX archive.$fExt", creates dArc if required pth_get_pArcArch #(pinn) of form "$dArc$fNoX archive.$fExt", creates dArc if required end with ' archive.' pth_get_byteSize #(pth)- res ipsa loquitor pth1_pth2_isEqual #(pth1, pth2) file [tMod, size, content, etc] are same pthL_fsizCompare_optr_process #(pth fsizeCompare optr)optr pthL based on fSizes (bytes) example : rm files <200 bytes (probably no corrections to make,rule of thumb only) pSortedL_rename_as_intL #() required by ffmpeg to produce video from image files of same format 04Aug2024 must make generic with input args, create zero-pad func in strings.sh still setup for jpg files... +-----+ SAFE [backup, restore] cart [pth, dir]s dir_archive_dirDated #(dir) archives ONLY files in dir pinn_arcArch #(pInn) NORMAL backup of form "$dArc$fNam archive.$fext", saves dated archive in 2nd (nested) z_Archive, creates z_Archive if doesn't exist pinn_arcArchRm #(pInn) archive then rm (clear) pinnL_arcArch #(pInn) normal archive of a list of pinn pinnL_arcArchRm #(pInn) archive then rm each of a list of pinn pBadL_archive_pArchiveL_restore #(pBadL) [archiveDated, restore archives as] current dInn_pInnL_arcReplace_dOvr #(dInn pSubL dOvr) use pInnL to [archive, replace] same files in dOvr wrkSpcL_arcArch #() easy backup of many workSpaces, see "$d_bin"'wrkSpcL_arcArch.sh' +-----+ SAFE [rm, rename] povrL, d_web transfer laptop[HP <-> TP]: dir_renameFileL #() rename an extracted list of files in dir using [sed, mv] povrL_rm #(povrL) DANGEROUS!! rm all files in povrL must check list to make sure it's safe to delete all! (even with flatArch) dFlatArchive a safety measure, but povrL_rm will over-write files with same fNam dWebHPtrans_rsync_dWebTP #() many [bash, rsync] tricks, especially fNam with [colon, hyphen] see "$d_bin"'rsync/rsync directories.sh' fNamPatn_findMakeRun_pCmdL #(dWork...) find fNamPatn files, check, make mv bash file, run comment-out section "read -p 'CHECK pCmdL!" to make sure pCmdL OK before over-writes! dOvr_pOvrL_arcReplaceWith_dInn #() pOvrL [archive, replaceFrom dInn] (eg dInn= a backup dir) pOvrL is with respect to dOvr, check pOvrL before proceeding! +-----+ timestamp changes - all time, dates are Unix timevalue unless otherwise specified date %s = seconds since the Epoch (1970-01-01 00:00 UTC), good for lftp uploads see "$d_bin"'strings.sh' to convert from one time format to another, eg (see list) pinn_get_timeAcs() #() for lftp uploads, access includes reads? pinn_get_timeChg #() timeChg from stat "$1" %s =seconds since the Epoch (1970-01-01 00:00 UTC), good for lftp uploads pinn_get_timeMod #() timeChg from stat "$1" pinn_getTimeModPut_pout #() set pout timeMod to that of pinn just use touch --reference="$pinn" "$pout" pinn_timeModTo_yymmddhms #() archive in current dir trailing date so fNam sorts alphabetically with function pinn_timeChgTo_yymmddhms #() archive in current dir trailing date so fNam sorts alphabetically with function +-----+ rename a selection of files in a dir dir_renameFileL #() rename an extracted list of files in dir using [sed, mv] +-----+ str in [fil, dir][,L] - find str, results to pout pout is often a temporary file for further processing pinn_pStrL_grep_dir #() find each str in pinn, results to dir, pNam "str.txt" dir_fextn_pthLExclud_findPthL_pout #() find str in files with fextn in dir, results to pout skips [file, dir]s, eg "z_Old\|z_Archive\|webSite maintain \|file_ops.ndf" shows only subDirFname of $dir385 str_getHowell_pout #() find str in Howell's [txt, html, email]s outputs [timeModUnix, tab, pth] -> sorted to pout pinnL_str_extractPthLWithStr_pout #() grep 'st' --with-file-name, lineNum must escape problematic chrs in str, pinnL_str_extractCountPathsWithStrTo_pout #() [txtLines with str of pinnL] write [str count, path]L to pout pinnL_pStrL_getPthsWithOUTstrL_pout #() paths of pinnL withOUT str write [str count, path]L to pout pInnL_pChrP_get_pOut #(pInnL pChrP pOut) extract strL with chr[start, end] reverse line each item pMain_strPthStrt_put_pOut #(strPthStrt, pMain, pOut) replace line strPthStrt with pOut 08********08 replace in povr[L]: [chrP, strOld_strNew, [p]StrP, pStrP] rpl [strfixed strP , regXpr no!, sedBadChrs OK , # of [strP huge, pth huge]] sed [hand-escaped strP , regXpr , sedBadChrs no!, # of [strP tiny, pth huge]] chrCde [auto-chrCde [povr, strP], regXpr yes, sedBadChrs OK , # of [strP huge, pth huge]] slow ..... do NOT use povr functions without first checking that 0_tests povr still work!!! best to do USBbackup before calling povrL functions for >3 povrs!!!! WARNING! pStrP etc must have ONE :Q: per line to separate str[Old, New], but NO others!!! +-----+ rpl replace in povr[L]: [strfixed strP, regXpr no!, sedBadChrs OK , # of [strP huge, pth huge]] povr_strOld_strNew_replaceRpl #(povr strP) basic povr function povr_strP_replaceRpl #(povr strP) povrL_pStrPL_replaceRpl #(bolBecho d_log povrL pStrP) dOvrL_pStrP_replaceRpl #(bolBecho d_log dOvrL pStrP) applies to all files in dOvrL not yet created, for now just see dOvrL_pStrP_replace_run() in 'fileops run.sh' +-----+ chrCode-based replace in povr[L]: povrL_pStrPL_replaceChrCde # see section below +-----+ sed replace in povr[L]: sed [hand-escaped strP, regXpr, sedBadChrs no!, # of [strP tiny, pth huge]] povr_strOld_strNew_replaceSed #(povr strP) basic povr function povr_strP_replaceSed #(povr strP) povrL_pStrPL_replaceSed #(bolBecho d_log povrL pStrP) dOvrL_pStrP_replaceSed #(bolBecho d_log dOvrL pStrP) applies to all files in dOvrL not yet created +-----+ chrCde replace in povr[L]: [auto-chrCde [povr, strP], regXpr yes, sedBadChrs OK, # of [strP huge, pth huge]] slow +--+ __This subsection is in 'strings.sh'. put here for easy reference : __#NOT chrSedBad?: `/ __chrSedBadEsy='&,.*\$^[]-' # escape-manageable (easy) __chrSedBadMid='&,.*\$^[]-{}|' # {}| are escape-problematic __chrSedBadAll='&,.*\$^[]-{}|' # use chrCde to avoid sed problems!! __ # very [slow, safe], missing {}|"' 03Jan2026 __chrGrepReplacOK='&*\$^[]-{}' __ __sed - convert between [human, code] forms of text __ Hum =human : human-readable, non-encoded format __ Cde =code : safe for sed expressions like "s|$strOld|$strNew|g" __ Apo =apostrophe ' : special "quote-enclosed" case, as apo can't be apo-enclosed __ Quo =quotation " : special "apostophe-enclosed" case, __ not used much here as almost all sed expressions are apo-enclosed __sedChrL_legend - chrL of special chrs that have to be encoded for sed __sed[x]To[y] - sedExpressions to [, de]-code chrSedL between [human, encoded] formats __strHum_sed_strCde #() convert str format [human -> code] __strCde_sed_strHum #() convert str format [code -> human] __str_strP_replace #() escape str[Old, New] then replace strOld with strNew in apo str __ str[, P] are escaped if bolChrCd == 1, returns same format as input +--+ p[Hum, Cde, ovr]_[chrP]_sed_p[Cde, Hum, ovr] search, replace] in pth[L], where strP of form "$strOld"$':Q:$strNew" pHum_sed_pCde #() convert pth format [human -> code], eg for pStrP pHum must NOT already be escaped! pCde is overwritten pCde_sed_pHum #() convert pth format [code -> human] pCde must NOT already be un-escaped! pCde is overwritten +-----+ povrL_pStrPL_replaceChrCde #(povrL pStrP) slow processing ({}| problematic) 08********08 pinn[,L]_strP_replace_[d, p]out (pinn strP pout) , encoded [strP, pStrP] parallel series to above this targets extraction of LINKS from webPages, for which the file locations not needed pinn_strP_replaceRpl_pout_sub #(pinn strP pout) pinn_strP_replaceSed_pout_sub #(pinn strP pout) pinn_strP_replaceRpl_pout #(pinn strP pout) sed replace, eg LINKS! pinn_strP_replaceSed_pout #(pinn strP pout) sed replace, eg LINKS! pinn_strP_replaceChrCde_pout #(pinn strP pout) replaceSed chrEncoded (strP, pinn) pinnL_pStrP_replaceRpl_dOut #(bolChrCd pinnL pStrP dOut)sed replace encoded (pStrP, pinnL) LINKS! +--+ __This subsection is in 'strings.sh' : __ pStrPL_rev "$pStrPL" # creates reversed pStrPL on startup __ strRev_sedStrtRev_sedEnddRev_get_pStrRevL #([str sed[Strt, Endd]]Rev) appends results to pStrRevL __ get multiple sedStr per line in strInn -> pLinkL __ one must REVERSE [str, sed[Strt, Endd]] because: __ the sequential cutting of strInn is based on "greedy sed", which finds the last occurence __ sedStart [] is NOT unique, and gives mixed up results (eg New] in povrs 08********08 [rgxP, pRgxP] sed [search, replace] in pth[L], rgxP of form "$rgxFind"$':Q:$rgxReplace" unlike strP, rgxP uses "pure sed" expressions that must NOT be encoded! uses triplicate form povr_rgxP_replace #(bolArXiv bolChrCd povr rgxP)sed replace encoded (rgxP, povr) put `\ in front of [()] - bash not geany povrL_pRgxP_replace #(bolArXiv bolChrCd povrL pRgxP)sed replace encoded (pRgxP, povrL) sed replaces each [strOld, strNew] in each povr, same pRgxP WARNING! all pRgxP cannot have :Q: other than to separate! put `\ in front of [()] - bash not geany povrL_pTypRgxT_replace #(type povrL pTypXprP) uses [(encoded) str, rgx] for sed type, either 'strP ' or 'rgxP ' trailing T in function name denotes "triple" (rather than pair) pTypXprT: strP of form "strOld\tstrNew" - expressions may [, not] be encoded rgxP of form "rgxFind\trgxReplace" - expressions must NOT be encoded! 08********08 lines - [extract, replace] * [line sequence, [func, optr] name]s] auto-create 'z_Archive' subDir if necessary -i option for [capital, small] insensitive - might not want this? forwardSlashes will likely be in str[Inn,Out], note that the `| c is used that could also cause problems, but hopefully not in paths! pinn_rmBlankLines_povr #(povr) removes [empty, with spaces] lines pinn_getSortU_pWordL #() get sorted list of words in txt file pinnL_lineIdx_strTst_split_pYes_pNo #(pinnL lineIdx strTst pYes pNo pDifYes pDifNo)- split pinnL to [pYes, pNo] according to strTst in line# lineIdx cut takes characters #1 through strTst_len333 (could be generalized ...) pinnL_getBadLine1_poutL #(pinnL poutL) list files with only one space 1st line '