#] #] ********************* #] "$d_Python"'0_python notes.txt' www.BillHowell.ca ~12Mar2022 initial To view this file - use a text editor (not word processor) constant width font (eg courrier 10), tab - 3 spaces use $ python3 and NOT $ python!!!!! Built-in Types: https://docs.python.org/3/library/stdtypes.html >> great reference Issues - see section below TableOfContents 48************************************************48 24************************24 # Table of Contents : # $ grep "^#]" "$d_Python"'0_python notes.txt' | sed 's/^#\]/ /' ********************* "$d_Python"'0_python notes.txt' Selected references : 10Mar2023 search 'python print a list' 09Mar2023 search 'Python print within def' 09Mar2023 Python indentation - immutable slavery 09Mar2023 search 'Python read a file as a list of line without return' don't read a file into memory, just iterate : for line in fileobject: 09Mar2023 search 'Python how do I load a program' $ python3 'pth.py' 09Mar2023 search 'Python read array' with open('filename.dat', newline='') as csvfile: 08Mar2023 search 'python while read file' 08Mar2023 search 'Python check if file exists' if os.path.isdir(prjDir): 07Mar2023 search 'Python while and do all lines have to be indented?' YES! same indentation 02Mar2023 search 'Python string catenate' ??? 02Mar2023 search 'python control loop example' 01Mar2023 Python tutorial 25Feb2023 The Python Package Index https://pypi.org/ 25Feb2023 search 'python neural networks' 25Feb2023 search 'python import global variable from file' 25Feb2023 importing-multiple-functions-from-a-python-module 25Feb2023 Built-in Types - PEP 3107 : def pick(l: list, index: int) -> int: 25Feb2023 import a file 25Feb2023 search 'python compiler' (Nuitka has acclaim) 25Feb2023 search 'python to c/c++ code converter' (Nuitka has acclaim) 25Feb2023 load python scripts 25Feb2023 Python source code libraries 25Feb2023 Convert [bash, QNial] libraries to Python - check if exist already! Howell practice, play Pure paths - Pure path objects provide path-handling operations which don’t actually access a filesystem. Correspondence [path, os] module tools (table) ??Mar2022 ruckus.convolution : how am I going to fix my sliding windows? (Howell videos) 23Mar2022 search "TypeError: Cannot create a consistent method resolution" 23Mar2022 install packages [KTBoost, ruckus] and a pile of Python3 modules 23Mar2022 python plt save mostly works, except curve labels, axis (labels, ticks, numbers) 22Mar2022 python plot problems 22Mar2022 finish python typeout of code 12Mar2022 type out code from images 24************************24 #] Selected references : Below is a table mapping various os functions to their corresponding PurePath/Path equivalent. Note : Not all pairs of functions/methods below are equivalent. Some of them, despite having some overlapping use-cases, have different semantics. They include os.path.abspath() and Path.absolute(), os.path.relpath() and PurePath.relative_to(). os and os.path pathlib os.path.abspath() Path.absolute() 1 os.path.realpath() Path.resolve() os.chmod() Path.chmod() os.mkdir() Path.mkdir() os.makedirs() Path.mkdir() os.rename() Path.rename() os.replace() Path.replace() os.rmdir() Path.rmdir() os.remove(), os.unlink() Path.unlink() os.getcwd() Path.cwd() os.path.exists() Path.exists() os.path.expanduser() Path.expanduser() and Path.home() os.listdir() Path.iterdir() os.path.isdir() Path.is_dir() os.path.isfile() Path.is_file() os.path.islink() Path.is_symlink() os.link() Path.hardlink_to() os.symlink() Path.symlink_to() os.readlink() Path.readlink() os.path.relpath() PurePath.relative_to() 2 os.stat() Path.stat(), Path.owner(), Path.group() os.path.isabs() PurePath.is_absolute() os.path.join() PurePath.joinpath() os.path.basename() PurePath.name os.path.dirname() PurePath.parent os.path.samefile() Path.samefile() os.path.splitext() PurePath.stem and PurePath.suffix os.chdir('C:/new/root') os.path.exists(path) https://docs.python.org/3/installing/index.html Installing Python Modules Key terms pip is the preferred installer program. Starting with Python 3.4, it is included by default with the Python binary installers. A virtual environment is a semi-isolated Python environment that allows packages to be installed for use by a particular application, rather than being installed system wide. venv is the standard tool for creating virtual environments, and has been part of Python since Python 3.3. Starting with Python 3.4, it defaults to installing pip into all created virtual environments. virtualenv is a third party alternative (and predecessor) to venv. It allows virtual environments to be used on versions of Python prior to 3.4, which either don’t provide venv at all, or aren’t able to automatically install pip into created environments. The Python Package Index is a public repository of open source licensed packages made available for use by other Python users. the Python Packaging Authority is the group of developers and documentation authors responsible for the maintenance and evolution of the standard packaging tools and the associated metadata and file format standards. They maintain a variety of tools, documentation, and issue trackers on both GitHub and Bitbucket. distutils is the original build and distribution system first added to the Python standard library in 1998. While direct use of distutils is being phased out, it still laid the foundation for the current packaging and distribution infrastructure, and it not only remains part of the standard library, but its name lives on in other ways (such as the name of the mailing list used to coordinate Python packaging standards development). https://pypi.org/project/labml-nn/ This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations, The website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better. >> looks good! https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/transformers/basic/autoregressive_experiment.ipynb Transformer Experiment This trains a simple transformer with multi headed attention introduced in Attention Is All You Need on an NLP auto-regression task (with Tiny Shakespeare dataset). https://pypi.org/project/neural-style/ This is a PyTorch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. The code is based on Justin Johnson's Neural-Style. The paper presents an algorithm for combining the content of one image with the style of another image using convolutional neural networks. Here's an example that maps the artistic style of The Starry Night onto a night-time photograph of the Stanford campus: >> awesome!! https://pypi.org/project/fast-transformer/ This repo implements Fastformer: Additive Attention Can Be All You Need by Wu et al. in TensorFlow. Fast Transformer is a Transformer variant based on additive attention that can handle long sequences efficiently with linear complexity. Fastformer is much more efficient than many existing Transformer models and can meanwhile achieve comparable or even better long text modeling performance. https://pypi.org/project/keras-transformer/ Transformer implemented in Keras Implementation of transformer for seq2seq tasks. https://pypi.org/project/transformer-pytorch/ A PyTorch implementation of Transformer from "Attention is All You Need" (https://arxiv.org/abs/1706.03762). This repo focuses on clean, readable, and modular implementation of the paper. 24************************24 08********08 #] ??Mar2023 08********08 #] ??Mar2023 08********08 #] ??Mar2023 08********08 #] ??Mar2023 08********08 #] ??Mar2023 08********08 #] 10Mar2023 search 'python print a list' +-----+ https://www.geeksforgeeks.org/print-lists-in-python-4-different-ways/ Print lists in Python (5 Different Ways) Difficulty Level : Easy Last Updated : 10 Jan, 2023 # Python program to print list # using for loop a = [1, 2, 3, 4, 5] # printing the list using loop for x in range(len(a)): print a[x], # printing the list using * operator separated # by comma print(*a) # printing the list using * and sep operator print("printing lists separated by commas") print(*a, sep = ", ") # print in new line print("printing lists in new line") print(*a, sep = "\n") # Python program to print list # by Converting a list to a # string for display a =["Geeks", "for", "Geeks"] # print the list using join function() print(' '.join(a)) # print the list by converting a list of # integers to string a = [1, 2, 3, 4, 5] print str(a)[1:-1] 08********08 #] 09Mar2023 search 'Python print within def' put code into def, indent with empty lines commented out def tester() L = [1,2,3,4] indexes = array('I', L) # print('L :',L) print('indexes',indexes) print('') # fileobject = open('/home/bill/web/CompLangs/Python/myPy/stdHdr.py') for line in fileobject: print(line.rstrip("\n")) $ python3 "$d_myPy"'8_test.py' File "/home/bill/web/CompLangs/Python/myPy/8_test.py", line 52 def tester() ^ SyntaxError: invalid syntax change def tester() to : def tester: >> nyet change def tester() to : def tester(): >> it runs, but no output! def tester(): L = [1,2,3,4] indexes = array('I', L) print('L :',L) print('indexes',indexes) print('') fileobject = open('/home/bill/web/CompLangs/Python/myPy/stdHdr.py') for line in fileobject: print(line.rstrip("\n")) >> same : it runs, but no output try L = [1,2,3,4] indexes = array('I', L) def tester(L, indexes): print('L :',L) print('indexes',indexes) print('') fileobject = open('/home/bill/web/CompLangs/Python/myPy/stdHdr.py') for line in fileobject: print(line.rstrip("\n")) tester(L, indexes) >> this worked!!! can't just use local variables for some reason? +-----+ https://stackoverflow.com/questions/5262264/learning-python-print-variable-inside-a-function-from-another-function Learning Python: print variable inside a function from another function Asked 12 years ago Modified 5 years, 2 months ago Viewed 23k times +-----+ https://stackoverflow.com/questions/51141395/how-to-print-inside-a-def-function how to print inside a def function Asked 4 years, 8 months ago Modified 4 years, 8 months ago Viewed 3k times +--+ Since you are returning the message from list_o_matic, you should just print the returning value from the caller: games = ['tlou', 'hitman', 'rainbow 6', 'nba2k'] print(games) def list_o_matic(inp): if inp == "": games.pop() return "the game " + inp + " was deleted" elif inp in games: games.remove(inp) return "the game " + inp + " was removed" elif inp != games: games.append(inp) return "the game " + inp + " was added" while True: if not games: print("goodbye!") break else: inp = input("write the name of the game: ") if inp == 'quit': print("goodbye!") break else: print(list_o_matic(inp)) print(games) Or, if you prefer to print the message inside the function as your title suggests, then print the message without returning it: games = ['tlou', 'hitman', 'rainbow 6', 'nba2k'] print(games) def list_o_matic(inp): if inp == "": games.pop() print("the game " + inp + " was deleted") elif inp in games: games.remove(inp) print("the game " + inp + " was removed") elif inp != games: games.append(inp) print("the game " + inp + " was added") while True: if not games: print("goodbye!") break else: inp = input("write the name of the game: ") if inp == 'quit': print("goodbye!") break else: list_o_matic(inp) print(games) answered Jul 2, 2018 at 17:59 blhsing +--+ Well your func2() is trying to print a variable in the scope of another function. You can either Return the value of var1 when calling func1 (eg. def func2(): print func1() } Call func2 from func1 and pass the value of var1 answered Mar 10, 2011 at 16:01 Dimitry +--+ You could try something like def func1(): var1 = "something" return var1 def func2(): print func1() If you need func1 to do more things than just define var1, then maybe what you need is to define a class and create objects? See http://docs.python.org/tutorial/classes.html answered Mar 10, 2011 at 16:08 Nick Edwards 08********08 #] 09Mar2023 Python indentation - immutable slavery +-----+ change : L = [1,2,3,4] indexes = array('I', L) to : L = [1,2,3,4] indexes = array('I', L) $ python3 "$d_myPy"'8_test.py' File "/home/bill/web/CompLangs/Python/myPy/8_test.py", line 53 indexes = array('I', L) IndentationError: unexpected indent >> wow, what a prick! +-----+ change : L = [1,2,3,4] indexes = array('I', L) to : L = [1,2,3,4] indexes = array('I', L) $ python3 "$d_myPy"'8_test.py' File "/home/bill/web/CompLangs/Python/myPy/8_test.py", line 52 L = [1,2,3,4] IndentationError: unexpected indent >> same shit! I need to use a Python null statement for alignments... 08********08 #] 09Mar2023 search 'Python read a file as a list of line without return' #] don't read a file into memory, just iterate : for line in fileobject: fileobject = open('/home/bill/web/CompLangs/Python/myPy/stdHdr.py') for line in fileobject: print(line) >> this works, but newline is appeanded to each line. How do I avoid that? I tried : print(line.rstrip("\n")) >> Bingo! it worked +-----+ https://stackoverflow.com/questions/13844162/list-of-lines-from-file-in-python List of lines from file in Python? Asked 10 years, 2 months ago Modified 10 years, 2 months ago Viewed 860 times +--+ You want the readlines method of a file object. fileobject = open(datafilename) lines = fileobject.readlines() Note that you (usually) don't need this. You can iterate over the file object directly and save yourself from having to store the whole file in memory: for line in fileobject: #do something with the line don't forget to close your fileobject when you're done! (context managers are quite helpful for that) Also, note that the lines will end with a newline ("\n"), but you can easily filter that off using .rstrip("\n") on the strings in the list or some variant in the str.strip family. e.g.: stripped_lines = [ line.rstrip("\n") for line in fileobject ] In other words, lines = fileobject.readlines() gives you the same thing as lines = list(fileobject) edited Dec 12, 2012 at 16:34 answered Dec 12, 2012 at 16:28 mgilson 08********08 #] 09Mar2023 search 'Python how do I load a program' $ python3 'pth.py' +-----+ https://realpython.com/run-python-scripts/ How to Run Your Python Scripts by Leodanis Pozo Ramos 33 Comments 08********08 #] 09Mar2023 search 'Python read array' with open('filename.dat', newline='') as csvfile: +-----+ https://stackoverflow.com/questions/14676265/how-to-read-a-text-file-into-a-list-or-an-array-with-python How to read a text file into a list or an array with Python Asked 10 years, 1 month ago Modified 1 year, 4 months ago Viewed 1.1m times +--+ You will have to split your string into a list of values using split() So, lines = text_file.read().split(',') EDIT: I didn't realise there would be so much traction to this. Here's a more idiomatic approach. import csv with open('filename.csv', 'r') as fd: reader = csv.reader(fd) for row in reader: # do something edited Sep 6, 2020 at 8:35 answered Feb 3, 2013 at 19:29 Achrome 3 I think that this answer could be bettered... If you consider a multiline .csv file (as mentioned by the OP), e.g., a file containing the alphabetic characters 3 by row (a,b,c, d,e,f, etc) and apply the procedure described above what you get is a list like this: ['a', 'b', 'c\nd', 'e', ... ] (note the item 'c\nd'). I'd like to add that, the above problem notwistanding, this procedure collapses data from individual rows in a single mega-list, usually not what I want when processing a record-oriented data file. – gboffi Jan 24, 2017 at 18:52 1 split is going to leave the newlines. Don't do this, use csv module or some other existing parser – Jean-François Fabre ♦ May 24, 2020 at 17:09 +--+ Better this way, def txt_to_lst(file_path): try: stopword=open(file_path,"r") lines = stopword.read().split('\n') print(lines) except Exception as e: print(e) answered Oct 21, 2021 at 7:39 Pranab Sarkar +--+ This question is asking how to read the comma-separated value contents from a file into an iterable list: 0,0,200,0,53,1,0,255,...,0. The easiest way to do this is with the csv module as follows: import csv with open('filename.dat', newline='') as csvfile: spamreader = csv.reader(csvfile, delimiter=',') Now, you can easily iterate over spamreader like this: for row in spamreader: print(', '.join(row)) See documentation for more examples. answered Mar 14, 2018 at 5:36 Blairg23 +--+ So you want to create a list of lists... We need to start with an empty list list_of_lists = [] next, we read the file content, line by line with open('data') as f: for line in f: inner_list = [elt.strip() for elt in line.split(',')] # in alternative, if you need to use the file content as numbers # inner_list = [int(elt.strip()) for elt in line.split(',')] list_of_lists.append(inner_list) A common use case is that of columnar data, but our units of storage are the rows of the file, that we have read one by one, so you may want to transpose your list of lists. This can be done with the following idiom by_cols = zip(*list_of_lists) Another common use is to give a name to each column col_names = ('apples sold', 'pears sold', 'apples revenue', 'pears revenue') by_names = {} for i, col_name in enumerate(col_names): by_names[col_name] = by_cols[i] so that you can operate on homogeneous data items mean_apple_prices = [money/fruits for money, fruits in zip(by_names['apples revenue'], by_names['apples_sold'])] Most of what I've written can be speeded up using the csv module, from the standard library. Another third party module is pandas, that lets you automate most aspects of a typical data analysis (but has a number of dependencies). Update While in Python 2 zip(*list_of_lists) returns a different (transposed) list of lists, in Python 3 the situation has changed and zip(*list_of_lists) returns a zip object that is not subscriptable. If you need indexed access you can use by_cols = list(zip(*list_of_lists)) that gives you a list of lists in both versions of Python. On the other hand, if you don't need indexed access and what you want is just to build a dictionary indexed by column names, a zip object is just fine... file = open('some_data.csv') names = get_names(next(file)) columns = zip(*((x.strip() for x in line.split(',')) for line in file))) d = {} for name, column in zip(names, columns): d[name] = column edited Aug 15, 2017 at 12:50 answered Nov 20, 2014 at 17:47 gboffi +-----+ https://stackoverflow.com/questions/52329167/how-to-read-python-array-from-file How to read Python array from file? [duplicate] Asked 4 years, 5 months ago Modified 4 years, 5 months ago Viewed 9k times +--+ I would like to write and read Python array from a txt file. I know to how to write it, but reading the array requires me to provide an argument about the length of the array. I do not the length of the array in advance, is there a way to read the array without computing its length. My work in progress code below. If I provide 1 as the second argument in a.fromfile, it reads the first element of the array, I would like all elements to read (basically the array to be recreated). from __future__ import division from array import array L = [1,2,3,4] indexes = array('I', L) with open('indexes.txt', 'w') as fileW: indexes.tofile(fileW) a = array('I', []) with open('indexes.txt', 'r') as fileR: a.fromfile(fileR,1) pythonarrays edited Sep 14, 2018 at 9:46 asked Sep 14, 2018 at 9:39 user58925 1 Did you come from a different programming language? because in python we don't need to define the size of the array, we can just create the list and we use list – Alleo Indong Sep 14, 2018 at 9:42 1 please also try to provide us the sample content of indexes.txt so we can check what seems to be the problem – Alleo Indong Sep 14, 2018 at 9:44 1 It would also be good if you explain why you want to use array rather than list. – PM 2Ring Sep 14, 2018 at 9:49 If you want to do this with binary data, I think the easy way to determine the array size is to compute it from the file size. There are funcfions in the os module that can get the file size efficiently. – PM 2Ring Sep 14, 2018 at 9:58 I use an array to save memory, I am working on a large problem – user58925 Sep 14, 2018 at 10:40 +--+ I don't know why .fromfile asks you to specify a number of objects but .frombytes doesn't. You can just read the bytes from the file and append them to the array. This works in python3: with open('indexes.txt', 'rb') as fileR: a.frombytes(fileR.read()) print(a) prints: array('I', [1, 2, 3, 4]) In python2 there is no .frombytes but .fromstring since str is equivalent to python3 bytes: with open('indexes.txt', 'rb') as fileR: a.fromstring(fileR.read()) edited Sep 14, 2018 at 10:44 answered Sep 14, 2018 at 10:26 Stop harming Monica 08********08 #] 08Mar2023 search 'python while read file' #] (can also read entire file as list of strings \n-terminated) +-----+ https://stackoverflow.com/questions/16957348/python-read-file-in-while-loop Python read file in while loop Asked 9 years, 9 months ago Modified 9 years, 1 month ago Viewed 10k times +--+ Why to write to a file and then read it ? Moreover you use flush, so you are doing potentially long io. I would do with open(originalpath) as f: mat = f.read() while condition : run += 1 write_mat_run(mat, run) mat = func(mat) write_mat_run may be done in another thread. You should check io exceptions. BTW this will probably solve your bug, or at least make it clear. answered Jun 6, 2013 at 9:03 MatthieuBizien +-----+ https://www.pythonforbeginners.com/files/4-ways-to-read-a-text-file-line-by-line-in-python 4 Ways to Read a Text File Line by Line in Python Author: Josh Petty Last Updated: May 27, 2021 Example 3: Reading files with a while loop and readline() file = open("wise_owl.txt",'r') while True: next_line = file.readline() if not next_line: break; print(next_line.strip()) file.close() 08********08 #] 08Mar2023 search 'Python check if file exists' if os.path.isdir(prjDir): +-----+ https://stackoverflow.com/questions/16842029/python-if-statement-with-exception-using-os-path-isdir Python: If statement with exception using os.path.isdir? Asked 9 years, 9 months ago Modified 9 years, 9 months ago Viewed 9k times Update: I think I misunderstood your question if you only want to check whether a directory exists or not then try this: def mk_prjDir(): print "Checking to see if", prjDir, "exists.." if os.path.isdir(prjDir): print (prjDir, "exists!") else: print (prjDir, "does not exist!") edited May 30, 2013 at 17:48 answered May 30, 2013 at 17:17 Ashwini Chaudhary +-----+ https://www.pythontutorial.net/python-basics/python-check-if-file-exists/ Python Check If File Exists import os.path os.path.exists() function from os.path import exists file_exists = exists(path_to_file) Path.is_file() method from pathlib import Path path = Path(path_to_file) path.is_file() +-----+ https://www.freecodecamp.org/news/how-to-check-if-a-file-exists-in-python/ January 5, 2023 How to Check if a File Exists in Python with isFile() and exists() Dionysia Lemonaki import os.path path = '/Users/dionysialemonaki/python_project/example.txt' print(os.path.isfile(file_path)) And as mentioned earlier, the isfile() method only works for files and not directories: import os.path path = './example.txt' check_file = os.path.exists(path) Then, create a new instance of the Path class and initialize it with the file path you want to check: from pathlib import Path # create a Path object with the path to the file path = Path('./example.txt') You can use the type() function to check the data type: from pathlib import Path path = Path('./example.txt') print(type(path)) # output is a pathlib object # This confirms that you created a Path object. 08********08 #] 07Mar2023 search 'Python while and do all lines have to be indented?' YES! same indentation +-----+ https://www.geeksforgeeks.org/indentation-in-python/ Indentation in Python Difficulty Level : Basic Last Updated : 13 Oct, 2022 Indentation is a very important concept of Python because without properly indenting the Python code, you will end up seeing IndentationError and the code will not get compiled. Python Indentation Python indentation refers to adding white space before a statement to a particular block of code. In another word, all the statements with the same space to the right, belong to the same code block. Most programming languages like C, C++, and Java use braces { } to define a block of code. Python uses indentation to highlight the blocks of code. 08********08 #] 02Mar2023 search 'Python string catenate' ??? +-----+ https://datagy.io/python-concatenate-strings/ How to Concatenate Strings in Python: A Complete Guide April 22, 2022 >> good summary Concatenating Strings in Python: Methods Compared The table below breaks down the different methods of concatenating strings in Python. Each of the methods is covered in more detail in the sections below, as well as some methods that didn’t make the list! Method Category f-strings Most readable + operator Fastest %s operator Backward compatible * operator Best to repeat lists Concatenating Strings in Python Using String Literals This final method to concatenate strings in Python is to use string literals in succession. By placing one string after another, Python implicitly joins these strings. Let’s take a look at an example of how this works: # Joining Strings with String Literals joined = "hey " "there" print(joined) # Returns: # hey there 08********08 #] 02Mar2023 search 'python control loop example' also search 'python if example' +-----+ https://www.dataquest.io/blog/tutorial-using-if-statements-in-python/ March 3, 2022 Tutorial: Using If Statements in Python +--+ # Nested if statements mark = 85 if mark >= 60 and mark <= 100: if mark >= 90: print("You are the best!") elif mark >= 80: print("Well done!") elif mark >= 70: print("You can do better.") else: print("Pass.") elif mark > 100: print("This mark is too high.") elif mark < 0: print("This mark is too low.") else: print("Failed.") +--+ Pattern matching with match..case syntax tomorrow = "snowy" match tomorrow: case "warm": print("I'll go to the sea.") case "very hot": print("I'll go to the forest.") case "snowy": print("I'll build a snowman.") case "rainy": print("I'll stay home.") case _: print("Weather not recognized.") # With pass num = 3 if num == 3: pass print("I'll write this code later.") +-----+ https://pythongeeks.org/loops-in-python/ Loops in Python with Examples >> great reference!! Example of Python while loop with else and break statement: i=7 while(i<=5): print(i) i=i+1 if(i==6): break else: print("This executes only when while does not loop at all") Example on while loop with else and break statement: num=5 while(num>0): print(num); num=num-1 +--+ 1. Python range function The range function is the most used for loops. So, let us discuss more on the range() function. a. Single argument: When we give an argument ‘n’ to the range(), then it creates the numbers from 0 to n-1. Example of Python range() function with one argument: range(10) list(range(10)) Output: range(0, 10) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] b. Two arguments: When we give two arguments (x,y) to the range(), then the sequence of the values from x to y-1 gets created. list(range(3,9)) Output: [3, 4, 5, 6, 7, 8] c. Three arguments: When we give two arguments (a,b,c) to the range(), then the sequence of the values from x to y-1 gets created. But between each element will have a difference/ a gap of ‘c’. Example of Python range() function with two arguments: list(range(1,20,3)) Output: [1, 4, 7, 10, 13, 16, 19] d. Negative values: Example of Python range() function with negative values: list(range(-2,5)) list(range(10,2,-3)) Output: [-2, -1, 0, 1, 2, 3, 4] [10, 7, 4] e. Getting empty sequences: Example of range() function giving empty sequences: list(range(4,2)) list(range(-2,-10,2)) list(range(-5,5,-2)) Output: [] [] [] +--+ 2. Looping over the iterables Example of iterating over list: list1=[1,'r',5.6,[3.2,'y'],7] for ele in list1: print(ele ) Output: 1 r 5.6 [3.2, ‘y’] 7 Example of iterating over set: set1={4,2,'y',0.5,'abc'} for ele in set1: print(ele ) Output: 0.5 2 4 y abc Example of iterating over tuple: tup=('a',4,'gfd',9.2) for ele in tup: print(ele) Output: a 4 gfd 9.2 Example of iterating over string: name='PythonGeeks' for char in name: print(char) Output: P y t h o n G e e k s Example of iterating over dictionary: dic={1:'r','y':3.4,6:[3,'t','r'],'e':0} for key in dic: print(key,":",dic[key]) Output: 1 : r y : 3.4 6 : [3, ‘t’, ‘r’] e : 0 08********08 #] 01Mar2023 Python tutorial +-----+ https://docs.python.org/3/tutorial/index.html 08********08 #] 25Feb2023 The Python Package Index https://pypi.org/ is a public repository of open source licensed packages made available for use by other Python users. https://pypi.org/ Probably all of the following have : 10,000+ projects with the selected classifier https://pypi.org/search/?q=neural+networks&o=&c=Programming+Language+%3A%3A+Python+%3A%3A+3 https://pypi.org/search/?q=support+vector+machines&o=&c=Programming+Language+%3A%3A+Python+%3A% https://pypi.org/search/?q=deep+learning&o=&c=Programming+Language+%3A%3A+Python+%3A%3A+3 samples of 10,000+ : https://pypi.org/project/deep-learning-bengio/ https://pypi.org/project/robust-deep-learning/ https://pypi.org/project/deep-expressions/ DeepExpressions is a Deep Learning toolkit for Facial Expressions Recognition (FER) which aims to enable an easy way to implement projects based on human emotions. https://pypi.org/project/deep-learning-for-multivariate-financial-time-series/ https://pypi.org/project/learning-deep-architectures-for-ai-zhongwenban-chuyi/ https://pypi.org/project/machine-learning-py/ mlpy - Machine Learning Py - High-Performance Python Package for Predictive Modeling https://pypi.org/project/pydbm/ pydbm` is Python library for building Restricted Boltzmann Machine(RBM), Deep Boltzmann Machine(DBM), Long Short-Term Memory Recurrent Temporal Restricted Boltzmann Machine(LSTM-RTRBM), and Shape Boltzmann Machine(Shape-BM). From the view points of functionally equivalents and structural expansions, this library also prototypes many variants such as Encoder/Decoder based on LSTM, Convolutional Auto-Encoder, and Spatio-temporal Auto-Encoder. https://github.com/accel-brain/accel-brain-code/tree/master/Deep-Learning-by-means-of-Design-Pattern https://pypi.org/project/PyFlowFW/ A deep learning framework https://pypi.org/project/Kylearn-pytorch/ A deep learning package https://pypi.org/project/trench/ Deep learning library https://pypi.org/project/tensorflow-intel/ TensorFlow is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms, and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team (https://research.google/teams/brain/) within Google's AI organization, TensorFlow comes with strong support for machine learning and deep learning with a flexible numerical computation core that can be used across many other scientific domains. TensorFlow is licensed under Apache 2.0. This optimized version of TensorFlow for Windows OS has been produced by Intel. In order to take full advantage of Intel architecture and to extract maximum performance, the TensorFlow framework has been optimized using oneAPI Deep Neural Network Library (oneDNN) primitives, a popular performance library for deep learning applications. For more information on the optimizations as well as performance data, see this blog post Faster AI Inference with Intel Optimization for TensorFlow. To activate oneDNN, execute the following in the Windows command prompt: set TF_ENABLE_ONEDNN_OPTS=1 before running your AI/ML workload. https://pypi.org/project/tf-settransformer/ An unofficial implementation of the Set Transformer framework in Tensorflow/Keras based on the paper: Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks. The official pytorch repository is available at: https://github.com/juho-lee/set_transformer. https://pypi.org/project/labml-nn/ This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations, The website renders these as side-by-side formatted notes. We believe these would help you understand these algorithms better. >> looks good! https://pypi.org/project/neural-style/ This is a PyTorch implementation of the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. The code is based on Justin Johnson's Neural-Style. The paper presents an algorithm for combining the content of one image with the style of another image using convolutional neural networks. Here's an example that maps the artistic style of The Starry Night onto a night-time photograph of the Stanford campus: >> awesome!! https://pypi.org/project/neural-learn/ A Deep Learning library based on Pytorch and PyTorch Lightning which makes Deep Learning even simpler. https://pypi.org/project/deepstruct/ deepstruct - neural network structure tool PyPI version Tests Documentation Status Downloads Python 3.8 Create deep neural networks based on very different kinds of graphs or use deepstruct to extract the structure of your deep neural network. Deepstruct combines tools for fusing machine learning and graph theory. We are fascinated with the interplay of end-to-end learnable, locally restricted models and their graph theoretical properties. Searching for evidence of the structural prior hypothesis. Interested in pruning, neural architecture search or learning theory in general? https://pypi.org/project/lean-transformer/ PyTorch transformers that don't hog your GPU memory. Memory-efficient transformers with optional sparsity, reversible layers, checkpoints, etc. https://pypi.org/project/momentumnet/ Official library for using Momentum Residual Neural Networks [1]. These models extend any Residual architecture (for instance it also work with Transformers) to a larger class of deep learning models that consume less memory. They can be initialized with the same weights as a pretrained ResNet and are promising in fine-tuning applications. https://pypi.org/project/deepclassifier/ DeepClassifier is a python package based on pytorch, which is easy-use and general for text classification task. You can install DeepClassifier by pip install -U deepclassifier。 If you want to know more information about DeepClassifier, please see the documentation. So let's start! https://pypi.org/project/neuroevo/ NeuroEvo is a platform for automated design and training of neural networks using evolutionary and particle swarm algorithms. The neuroevo Python package implements the algorithms used in the web application (neuroevo.io). Preprint: https://arxiv.org/abs/2210.00286 Developed by Philip Schroeder (pschroe9@jhu.edu, pschroed@broadinstitute.org) https://pypi.org/project/fast-transformer/ This repo implements Fastformer: Additive Attention Can Be All You Need by Wu et al. in TensorFlow. Fast Transformer is a Transformer variant based on additive attention that can handle long sequences efficiently with linear complexity. Fastformer is much more efficient than many existing Transformer models and can meanwhile achieve comparable or even better long text modeling performance. https://pypi.org/project/keras-transformer/ Transformer implemented in Keras Implementation of transformer for seq2seq tasks. 08********08 #] 25Feb2023 search 'python neural networks' +-----+ https://realpython.com/python-ai-neural-network/ Python AI: How to Build a Neural Network & Make Predictions by Déborah Mesquita 08********08 #] 25Feb2023 search 'python import global variable from file' +-----+ https://www.geeksforgeeks.org/how-to-import-variables-from-another-file-in-python/ How to import variables from another file in Python? Example: Suppose we have a file named “swaps.py”. We have to import the x and y variable from this file in another file named “calval.py”. # swaps.py file from which variables to be imported x = 23 y = 30 def swapVal(x, y): x,y = y,x return x, y Now create a second python file to call the variable from the above code: # calval.py file where to import variables # import swaps.py file from which variables # to be imported # swaps.py and calval.py files should be in # same directory. import swaps # Import x and y variables using # file_name.variable_name notation new_x = swaps.x new_y = swaps.y print("x value: ", new_x, "y value:", new_y) # Similarly, import swapVal method from swaps file x , y = swaps.swapVal(new_x,new_y) print("x value: ", x, "y value:", y) Output: x value: 23 y value: 30 x value: 30 y value: 23 +-----+ https://instructobit.com/tutorial/108/How-to-share-global-variables-between-files-in-Python How to share global variables between files in Python Learn how to create global variables that can be shared between files/modules in Python. Christopher Thornton @Instructobit access_time Modified: 2 years ago Sharing global variables between files/modules in Python Even though the global keyword is conventionally only used to access variables from within the same module, there are ways to use it in order to share variables between files. For example, we could take influence from PHP and create something similar to the "superglobals" variables. To do this, you can create a new module specifically for storing all the global variables your application might need. For this you can create a function that will initialize any of these globals with a default value, you only need to call this function once from your main class, then you can import the globals file from any other class and use those globals as needed. +-----+ https://www.freecodecamp.org/news/python-global-variables-examples/ May 12, 2022 Python Global Variables – How to Define a Global Variable Example Dionysia Lemonaki >> good reference, but not good enough 08********08 #] 25Feb2023 importing-multiple-functions-from-a-python-module +-----+ https://stackoverflow.com/questions/6761825/importing-multiple-functions-from-a-python-module importing multiple functions from a Python module Asked 11 years, 7 months ago Modified 2 months ago Viewed 108k times Q: I am importing lots of functions from a module Is it better to use from my_module import function1, function2, function3, function4, function5, function6, function7 which is a little messy, but avoids flooding the current namespace with everything from that module or from my_module import * Which looks tidy but will fill the namespace with everything from that module. Can't find anything in PEP8 about what the limit for how much you should import by name is. Which is better and why? asked Jul 20, 2011 at 12:23 Jim Jeffries 08********08 #] 25Feb2023 Built-in Types - PEP 3107 : def pick(l: list, index: int) -> int: +-----+ https://docs.python.org/3/library/stdtypes.html Built-in Types >> great reference Truth Value Testing Boolean Operations — and, or, not Comparisons - < <= > >= == != is is not Numeric Types — int, float, complex Iterator Types Sequence Types — list, tuple, range Text Sequence Type — str class str(object='') class str(object=b'', encoding='utf-8', errors='strict') Binary Sequence Types — bytes, bytearray, memoryview Set Types — set, frozenset Type Annotation Types — Generic Alias, Union Other Built-in Types Integer string conversion length limitation +-----+ https://stackoverflow.com/questions/2489669/how-do-python-functions-handle-the-types-of-parameters-that-you-pass-in How do Python functions handle the types of parameters that you pass in? Asked 12 years, 11 months ago Modified 8 months ago Viewed 727k times +--+ The other answers have done a good job at explaining duck typing and the simple answer by tzot: Python does not have variables, like other languages where variables have a type and a value; it has names pointing to objects, which know their type. However, one interesting thing has changed since 2010 (when the question was first asked), namely the implementation of PEP 3107 (implemented in Python 3). You can now actually specify the type of a parameter and the type of the return type of a function like this: def pick(l: list, index: int) -> int: return l[index] Here we can see that pick takes 2 parameters, a list l and an integer index. It should also return an integer. So here it is implied that l is a list of integers which we can see without much effort, but for more complex functions it can be a bit confusing as to what the list should contain. We also want the default value of index to be 0. To solve this you may choose to write pick like this instead: def pick(l: "list of ints", index: int = 0) -> int: return l[index] Note that we now put in a string as the type of l, which is syntactically allowed, but it is not good for parsing programmatically (which we'll come back to later). It is important to note that Python won't raise a TypeError if you pass a float into index, the reason for this is one of the main points in Python's design philosophy: "We're all consenting adults here", which means you are expected to be aware of what you can pass to a function and what you can't. If you really want to write code that throws TypeErrors you can use the isinstance function to check that the passed argument is of the proper type or a subclass of it like this: def pick(l: list, index: int = 0) -> int: if not isinstance(l, list): raise TypeError return l[index] More on why you should rarely do this and what you should do instead is talked about in the next section and in the comments. PEP 3107 does not only improve code readability but also has several fitting use cases which you can read about here. Previously when one documented Python code with for example Sphinx some of the above functionality could be obtained by writing docstrings formatted like this: def pick(l, index): """ :param l: list of integers :type l: list :param index: index at which to pick an integer from *l* :type index: int :returns: integer at *index* in *l* :rtype: int """ return l[index] As you can see, this takes a number of extra lines (the exact number depends on how explicit you want to be and how you format your docstring). But it should now be clear to you how PEP 3107 provides an alternative that is in many (all?) ways superior. This is especially true in combination with PEP 484 which, as we have seen, provides a standard module that defines a syntax for these type hints/annotations that can be used in such a way that it is unambiguous and precise yet flexible, making for a powerful combination. In my personal opinion, this is one of the greatest features in Python ever. I can't wait for people to start harnessing the power of it. Sorry for the long answer, but this is what happens when I get excited. 08********08 #] 25Feb2023 import a file +-----+ https://stackoverflow.com/questions/15681366/importing-files-in-python Importing files in Python? Asked 9 years, 11 months ago Modified 4 years, 1 month ago Viewed 54k times 1st option: Add the path to your files to the default paths Pythons looks at. import sys sys.path.insert(0, 'C:/complete/path/to/my/directory') 2nd option: Add the path relative to your current root of your environment (current directory), using instead the following: #Learn your current root import os os.getcwd() #Change your current root (optional) os.chdir('C:/new/root') #Add the path from your current root '.' to your directory import sys sys.path.insert(0, './Path/from/current/root/to/my/directory') edited Dec 30, 2018 at 3:51 answered Dec 30, 2018 at 3:38 Sarah Labbé >> I tried sys.path.insert(0, '/home/bill/web/Software programming & code/Python/myPy/') 08********08 #] 25Feb2023 search 'python compiler' (Nuitka has acclaim) +-----+ https://nuitka.net/doc/user-manual.html Nuitka User Manual Overview This document is the recommended first read if you are interested in using Nuitka, understand its use cases, check what you can expect, license, requirements, credits, etc. Nuitka is the Python compiler. It is written in Python. It is a seamless replacement or extension to the Python interpreter and compiles every construct that CPython 2.6, 2.7, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, 3.9, 3.10 have, when itself run with that Python version. It then executes uncompiled code and compiled code together in an extremely compatible manner. You can use all Python library modules and all extension modules freely. Nuitka translates the Python modules into a C level program that then uses libpython and static C files of its own to execute in the same way as CPython does. All optimization is aimed at avoiding overhead, where it’s unnecessary. None is aimed at removing compatibility, although slight improvements will occasionally be done, where not every bug of standard Python is emulated, e.g. more complete error messages are given, but there is a full compatibility mode to disable even that. Install Nuitka python -m pip install nuitka Verify using command python -m nuitka --version >>> python -m pip install nuitka File "", line 1 python -m pip install nuitka ^ SyntaxError: invalid syntax +--+ You can compile Python scripts to a binary code using various methods, but I have found out that using Nuitka is more efficient. Nuitka is a Python-to-C++ compiler that supports almost all versions of python. The command syntax is as easy as nuitka hello.py Goto http://nuitka.net/doc/user-manual.html for more information. edited Aug 25, 2018 at 2:58 user258532 answered Jan 30, 2017 at 23:37 salafi 7 More efficient how? Why would anyone want to use it? – muru Jan 31, 2017 at 0:02 nuitka is indeed very useful - for example, it allows you to ship Python programs to computers where Python is not installed. But as nuitka compiles Python programs to C++ machine code, you can circumvent that. – user258532 Aug 24, 2018 at 13:44 +-----+ https://askubuntu.com/questions/324871/how-to-compile-a-python-file How to compile a python file? Asked 9 years, 7 months ago Modified 3 years, 1 month ago Viewed 729k times +--+ You may also try compileall: python -m compileall ./ edited Dec 1, 2015 at 8:53 Jens Erat answered Nov 30, 2015 at 16:21 MediaVince https://docs.python.org/3/library/compileall.html – myrdd Aug 14, 2019 at 13:24 +-----+ https://docs.python.org/3/library/compileall.html compileall — Byte-compile Python libraries Source code: Lib/compileall.py >> not useful, as I don't want to build a compiler!! 08********08 #] 25Feb2023 search 'python to c/c++ code converter' (Nuitka has acclaim) +-----+ https://stackoverflow.com/questions/4650243/convert-python-program-to-c-c-code Convert Python program to C/C++ code? [closed] Asked 12 years, 1 month ago Modified 4 months ago Viewed 531k times +--+ Yes. Look at Cython. It does just that: Converts Python to C for speedups. edited Jan 10, 2011 at 18:56 answered Jan 10, 2011 at 18:49 Lennart Regebro 10 Of course that won't save you anything unless you add a bunch of cdef declarations and thereby introduce static typing (otherwise you just juggle opaque PyObject * stuff). And it will never get quite as fast as plain C because it's usually interfacing with Python (100% or more? only for plain numerical code that doesn't interface with Python at all for the most time!). But other than that, yes, it can get you a pretty devent speedup. – user395760 Jan 10, 2011 at 19:00 9 @delnan: In fact, it does save you something. Most pure Python code will be faster after compiled. But yes, with the cdefs and static typing you really start seeing differences. And the interfacing with Python you get in all cases where you use C from Python. – +--+ Shed Skin is "a (restricted) Python-to-C++ compiler". From the docs: Shed Skin is an experimental compiler, that can translate pure, but implicitly statically typed Python (2.4-2.6) programs into optimized C++. It can generate stand-alone programs or extension modules that can be imported and used in larger Python programs. Besides the typing restriction, programs cannot freely use the Python standard library (although about 25 common modules, such as random and re, are currently supported). Also, not all Python features, such as nested functions and variable numbers of arguments, are supported. For a set of a 75 non-trivial programs (at over 25,000 lines in total (sloccount)), measurements show a typical speedup of 2-200 times over CPython. edited Oct 26, 2022 at 15:14 Burak answered Jan 10, 2011 at 19:08 ephemient 3 +1 one advantage of Shed Skin is type inference: if it is possible to go guess variable types from the program flow, dynamic type-checking is avoided. This typically leads to shorter C++ code that it is actually possible to read and compiles to faster programs. – Kyss Tao Mar 30, 2012 at 14:19 1 There is also Python → 11l → C++ transpiler, which is also a restricted Python to C++ compiler, but it supports some Python features, which is not supported with Shed Skin (e.g. nested functions/closures). – tav Dec 11, 2018 at 10:27 +--+ Just came across this new tool in hacker news. http://nuitka.net/pages/overview.html From their page - "Nuitka is a good replacement for the Python interpreter and compiles every construct that CPython 2.6, 2.7, 3.2 and 3.3 offer. It translates the Python into a C++ program that then uses "libpython" to execute in the same way as CPython does, in a very compatible way." edited May 5, 2014 at 18:08 Jordan.J.D answered May 5, 2014 at 17:50 seagull1089 2 This project is so much more mature than other similar options. It's funny it creates the binary with an .exe extension on OSX even though it is a perfectly normal OSX Mach-O executable. Looks like it might be a good replacement for pyinstaller, py2exe, py2app, etc. The --recurse-*** flags are important to set properly though. – ccpizza Jul 24, 2016 at 8:23 2 Nuitka is great, but the C/C++ code created is using PyObject which bind to the CPython-C-code-implementation. It does not produce idiomatic C-code. – Make42 Mar 15, 2020 at 11:33 +--+ I know this is an older thread but I wanted to give what I think to be helpful information. I personally use PyPy which is really easy to install using pip. I interchangeably use Python/PyPy interpreter, you don't need to change your code at all and I've found it to be roughly 40x faster than the standard python interpreter (Either Python 2x or 3x). I use pyCharm Community Edition to manage my code and I love it. I like writing code in python as I think it lets you focus more on the task than the language, which is a huge plus for me. And if you need it to be even faster, you can always compile to a binary for Windows, Linux, or Mac (not straight forward but possible with other tools). From my experience, I get about 3.5x speedup over PyPy when compiling, meaning 140x faster than python. PyPy is available for Python 3x and 2x code and again if you use an IDE like PyCharm you can interchange between say PyPy, Cython, and Python very easily (takes a little of initial learning and setup though). Some people may argue with me on this one, but I find PyPy to be faster than Cython. But they're both great choices though. Edit: I'd like to make another quick note about compiling: when you compile, the resulting binary is much bigger than your python script as it builds all dependencies into it, etc. But then you get a few distinct benefits: speed!, now the app will work on any machine (depending on which OS you compiled for, if not all. lol) without Python or libraries, it also obfuscates your code and is technically 'production' ready (to a degree). Some compilers also generate C code, which I haven't really looked at or seen if it's useful or just gibberish. Good luck. Hope that helps. edited Apr 26, 2019 at 22:16 answered May 12, 2016 at 16:03 jacktrader What software do you use to compile from PyPy interpretation? – Vasyl Vaskivskyi Oct 9, 2019 at 15:45 Not specifically PyPy, just .py scripts. Nuitka if you want "C/C++ executable's, or C/C++ source code" and PyInstaller if you just want an executable (easier). There's also py2exe but I had less success with it, though I'm sure things have improved. PyInstaller is also cross platform, not just for Windows executable's (works with Linux, and Mac). Nuitka is unique because I think it's the only "compiler" that gives you usable source code back that you could in theory optimize further. There are a few others like bbFreeze, cx_Freeze, and py2app but I haven't tried them. Best of luck! – jacktrader Oct 9, 2019 at 17:30 1 I have also found PyPy to run faster than Cython. In one test I actually found PyPy to be the same speed as a C++ version of the program (insertion sort). – Nv7 May 17, 2020 at 3:11 Add a comment 10 +--+ Another option - to convert to C++ besides Shed Skin - is Pythran. To quote High Performance Python by Micha Gorelick and Ian Ozsvald: Pythran is a Python-to-C++ compiler for a subset of Python that includes partial numpy support. It acts a little like Numba and Cython—you annotate a function’s arguments, and then it takes over with further type annotation and code specialization. It takes advantage of vectorization possibilities and of OpenMP-based parallelization possibilities. It runs using Python 2.7 only. One very interesting feature of Pythran is that it will attempt to automatically spot parallelization opportunities (e.g., if you’re using a map), and turn this into parallel code without requiring extra effort from you. You can also specify parallel sections using pragma omp > directives; in this respect, it feels very similar to Cython’s OpenMP support. Behind the scenes, Pythran will take both normal Python and numpy code and attempt to aggressively compile them into very fast C++—even faster than the results of Cython. You should note that this project is young, and you may encounter bugs; you should also note that the development team are very friendly and tend to fix bugs in a matter of hours. answered Apr 19, 2016 at 12:30 boardrider 2 Note that "until 0.9.5 (included), Pythran was supporting Python 3 and Python 2.7. It now only supports Python 3." – Make42 Aug 24, 2020 at 17:12 08********08 #] 25Feb2023 load python scripts +-----+ https://realpython.com/run-python-scripts/ How to Run Your Python Scripts by Leodanis Pozo Ramos 33 Comments >> great reference Running Modules With the -m Option Python offers a series of command-line options that you can use according to your needs. For example, if you want to run a Python module, you can use the command python -m . The -m option searches sys.path for the module name and runs its content as __main__: $ python3 -m hello Hello World! Note: module-name needs to be the name of a module object, not a string. 08********08 #] 25Feb2023 Python source code libraries see "$d_web"'Software programming & code/Python/1_python source code listings.txt' for path in sys.path: print(path) for path in sys.path: print(path) >>> for path in sys.path: ... print(path) File "", line 2 print(path) ^ IndentationError: expected an indented block >> must have an indent ` or \t >> I needed import sys >>> import sys >>> for path in sys.path: ... print(path) ... /usr/lib/python39.zip /usr/lib/python3.9 /usr/lib/python3.9/lib-dynload /usr/local/lib/python3.9/dist-packages /usr/lib/python3/dist-packages /usr/lib/python3.9/dist-packages >> OK +-----+ https://www.python.org/about/gettingstarted/ Python For Beginners >> nothing much - links to other sites +-----+ search 'python and how do I load a program?' https://realpython.com/run-python-scripts/ How to Run Your Python Scripts by Leodanis Pozo Ramos 33 Comments Scripts vs Modules So, the main difference between a module and a script is that modules are meant to be imported, while scripts are made to be directly executed. 08********08 #] 25Feb2023 Convert [bash, QNial] libraries to Python - check if exist already! search 'Python file management libraries' +-----+ #] Howell practice, play +--+ https://docs.python.org/3/library/pathlib.html#basic-use >>> from pathlib import Path >>> p = Path('.') >>> [x for x in p.iterdir() if x.is_dir()] [PosixPath('.hg'), PosixPath('docs'), PosixPath('dist'), PosixPath('__pycache__'), PosixPath('build')] Traceback (most recent call last): File "", line 1, in NameError: name 'PosixPath' is not defined >> I have to load PurePath first? >>> PurePath('setup.py') # Running on a Unix machine Traceback (most recent call last): File "", line 1, in NameError: name 'PurePath' is not defined >>> from pathlib import PurePath >>> PurePath('setup.py') PurePosixPath('setup.py') >> OK, now things work >>> list(p.glob('**/*.py')) >> HUGE list of files as one big strL, eg [... PosixPath('PROJECTS/backup_temp/RaspPi_ext4_32Gb/SysMaint SWAPPER/Program installation & setup files/jahshaka-0.1.0/jahplayer/setup.py'), ...] I get the same results as the webPage Navigating inside a directory tree: >>> p = Path('/etc') >>> q = p / 'init.d' / 'reboot' >>> q PosixPath('/etc/init.d/reboot') >>> q.resolve() PosixPath('/etc/rc.d/init.d/halt') +--+ #] Pure paths - Pure path objects provide path-handling operations which don’t actually access a filesystem. There are three ways to access these classes, which we also call flavours: class pathlib.PurePath(*pathsegments) A generic class that represents the system’s path flavour (instantiating it creates either a PurePosixPath or a PureWindowsPath): >>> PurePath('setup.py') # Running on a Unix machine PurePosixPath('setup.py') >> lots of great learning examples +-----+ #] Correspondence [path, os] module tools (table) Below is a table mapping various os functions to their corresponding PurePath/Path equivalent. Note : Not all pairs of functions/methods below are equivalent. Some of them, despite having some overlapping use-cases, have different semantics. They include os.path.abspath() and Path.absolute(), os.path.relpath() and PurePath.relative_to(). os and os.path pathlib os.path.abspath() Path.absolute() 1 os.path.realpath() Path.resolve() os.chmod() Path.chmod() os.mkdir() Path.mkdir() os.makedirs() Path.mkdir() os.rename() Path.rename() os.replace() Path.replace() os.rmdir() Path.rmdir() os.remove(), os.unlink() Path.unlink() os.getcwd() Path.cwd() os.path.exists() Path.exists() os.path.expanduser() Path.expanduser() and Path.home() os.listdir() Path.iterdir() os.path.isdir() Path.is_dir() os.path.isfile() Path.is_file() os.path.islink() Path.is_symlink() os.link() Path.hardlink_to() os.symlink() Path.symlink_to() os.readlink() Path.readlink() os.path.relpath() PurePath.relative_to() 2 os.stat() Path.stat(), Path.owner(), Path.group() os.path.isabs() PurePath.is_absolute() os.path.join() PurePath.joinpath() os.path.basename() PurePath.name os.path.dirname() PurePath.parent os.path.samefile() Path.samefile() os.path.splitext() PurePath.stem and PurePath.suffix +-----+ https://docs.python.org/3/library/filesys.html File and Directory Access The modules described in this chapter deal with disk files and directories. For example, there are modules for reading the properties of files, manipulating paths in a portable way, and creating temporary files. The full list of modules in this chapter is: pathlib — Object-oriented filesystem paths Basic use Pure paths General properties Operators Accessing individual parts Methods and properties Concrete paths Methods Correspondence to tools in the os module os.path — Common pathname manipulations fileinput — Iterate over lines from multiple input streams stat — Interpreting stat() results filecmp — File and Directory Comparisons The dircmp class tempfile — Generate temporary files and directories Examples Deprecated functions and variables glob — Unix style pathname pattern expansion fnmatch — Unix filename pattern matching linecache — Random access to text lines shutil — High-level file operations Directory and files operations Platform-dependent efficient copy operations copytree example rmtree example Archiving operations Archiving example Archiving example with base_dir Querying the size of the output terminal 08********08 #] ddMar2022 ruckus.convolution : how am I going to fix my sliding windows? (Howell videos) ImportError: cannot import name 'sliding_window_view' see previous search "numpy.lib.stride_tricks _sliding_window_view" +-----+ https://numpy.org/devdocs/reference/generated/numpy.lib.stride_tricks.sliding_window_view.html Notes For many applications using a sliding window view can be convenient, but potentially very slow. Often specialized solutions exist, for example: scipy.signal.fftconvolve filtering functions in scipy.ndimage moving window functions provided by bottleneck. As a rough estimate, a sliding window approach with an input size of N and a window size of W will scale as O(N*W) where frequently a special algorithm can achieve O(N). That means that the sliding window variant for a window size of 100 can be a 100 times slower than a more specialized version. Nevertheless, for small window sizes, when no custom algorithm exists, or as a prototyping and developing tool, this function can be a good solution. +-----+ convolution.py - I switched comment-out to : from numpy.lib.stride_tricks import sliding_window_view as _sliding_window_view # from numpy import sliding_window_view as _sliding_window_view Retry, same error : ImportError: cannot import name 'sliding_window_view' https://github.com/twopirllc/pandas-ta/issues/285 https://stackoverflow.com/questions/55091454/cannot-create-a-consistent-method-resolution-order-inheriting-from-another-chi 08********08 #] 23Mar2022 search "TypeError: Cannot create a consistent method resolution" >>> import ruckus_Lorenz_attractor Traceback (most recent call last): File "", line 1, in File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus_Lorenz_attractor.py", line 27, in import ruckus File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/__init__.py", line 5, in from ruckus.embedding import EigenRKHS File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/embedding.py", line 15, in class EigenRKHS(_KernelPCA,_RKHS): TypeError: Cannot create a consistent method resolution order (MRO) for bases TransformerMixin, BaseEstimator +-----+ https://stackoverflow.com/questions/55091454/cannot-create-a-consistent-method-resolution-order-inheriting-from-another-chi >> 3 great answers +-----+ embedding.py line 15 - first line of class definitions all had : r""" -> I removed the r (probably no difference) - class EigenRKHS(_KernelPCA,_RKHS): -> # class EigenRKHS(_RKHS): - comma after last item in list for two class defs convolution.py line 61 - comma after last item in list +-----+ >>> import ruckus_Lorenz_attractor Traceback (most recent call last): File "", line 1, in File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus_Lorenz_attractor.py", line 27, in import ruckus File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/__init__.py", line 11, in import ruckus.convolution as convolution File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/convolution.py", line 4, in from numpy.lib.stride_tricks import sliding_window_view as _sliding_window_view ImportError: cannot import name 'sliding_window_view' http://numpy.org YIKES!! this is a recently-reported 06May2021 bug in numpy! https://github.com/twopirllc/pandas-ta/issues/285 twopirllc commented on Jul 28, 2021 Hello @AlexandreBourrieau, @brandonculver, @dikien, @opsxcq, @peturssonstefan, @rushiai, New stable version v0.3.14b should help resolve this Issue. Please let me know otherwise. $ pip install pandas_ta Thanks, KJ >> Software Manager install of : Python3-pandas - data structures for "relational" or :labelled" data Python3-geopandas - add support for geographical data [GeoSeries, GeoDataFrame] type to pandas objects avoid sliding_window problem by commenting out # import ruckus.convolution as convolution in "$d_Python"'ruckus/ruckus/__init__.py' +-----+ >>> import ruckus_Lorenz_attractor Traceback (most recent call last): File "", line 1, in File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus_Lorenz_attractor.py", line 27, in import ruckus File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/__init__.py", line 8, in import ruckus.sampling as sampling File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/sampling.py", line 48 """ SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 484-485: truncated \uXXXX escape >> I split comments in two with """ - hopefully short enough, if that was the problem >>Retry (always same error, different line): >> problem line 23 now - end of first half of mui-line comment? Keep splitting in two until I find it File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/sampling.py", line 23 File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/sampling.py", line 26 File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/sampling.py", line 29 File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/sampling.py", line 28 File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/sampling.py", line 31 >> problem line : x_{t} = \underset{x}{\mathrm{argmax}}\left<\phi(x),w_t\right> \\ > remove extra comment splits & retestto confirm File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus/sampling.py", line 25 >> OK, so problem line is : x_{t} = \underset{x}{\mathrm{argmax}}\left<\phi(x),w_t\right> \\ I removed it - now loads. +-----+ >>> import ruckus_Lorenz_attractor Traceback (most recent call last): File "", line 1, in File "/media/bill/Dell2/Website - raw/Software programming & code/Python/ruckus/ruckus_Lorenz_attractor.py", line 62, in conv_net = ruckus.convolution.ConvolutionalRKHS( AttributeError: module 'ruckus' has no attribute 'convolution' This bngs me back to : avoid sliding_window problem by commenting out # import ruckus.convolution as convolution in "$d_Python"'ruckus/ruckus/__init__.py' So re-instate ruckus.convolution. Pain in the ass, how am I going to fix my sliding windows? 08********08 #] 23Mar2022 install packages [KTBoost, ruckus] and a pile of Python3 modules I first installed python [pip, wheel] yikes! need to L Software Manager install python3 [numpy, matplotlib.pyplot, etc] : Python3-bottleneck - http://berkeleyanalytics.com/bottleneck collection of fast NumPy array functions written in C, [int, float] cart [32,64] only Python3-numexpr - fast numerical array expression evaluator for Python3 and NumPy Python3-multipleau - http://paulmeuller.github.io/multipletau correlation is computed on a logarithmic scale (less data points are computed) and is thus much faster than conventional correlation on a linear scale such as 'numpy.correlate' Python3-numpy - fast array facility to the Python3 language Python2-matplotlib - Python based pllotting system in a style similar to MatLab Python3-tk - Tkinter - writing Tk applications with Python3.x Python3-nltk - libraries for natural language processing, It provides easy-to-use intterfaces to over 50 corpora and exical resources such as WordNet, along with a suit of text processing libraries for [classification, tokenization, stemming, tagging, parsing, semantic reasoning] Python3-sklearn - modules for machine learning and data mining Python3-sklearn-lib - low-level implementations and bindigs for scikit-learn 08********08 #] 23Mar2022 python plt save mostly works, except curve labels, axis (labels, ticks, numbers) 23Mar2022 I have to exit() python restart for each run. Why?? 08********08 #] 22Mar2022 python plot problems +-----+ Save the plot (don't show) : https://matplotlib.org/stable/gallery/misc/svg_filter_line.html # save the figure as a bytes string in the svg format. f = io.BytesIO() plt.savefig(f, format="svg") ... coding # read in the saved svg tree, xmlid = ET.XMLID(f.getvalue()) # insert the filter definition in the svg dom tree. tree.insert(0, ET.XML(filter_def)) for l in [l1, l2]: # pick up the svg element with given id shadow = xmlid[l.get_label() + "_shadow"] # apply shadow filter shadow.set("filter", 'url(#dropshadow)') fn = "svg_filter_line.svg" print(f"Saving '{fn}'") ET.ElementTree(tree).write(fn) +-----+ https://stackoverflow.com/questions/458209/is-there-a-way-to-detach-matplotlib-plots-so-that-the-computation-can-continue Use matplotlib's calls that won't block: Using draw(): from matplotlib.pyplot import plot, draw, show plot([1,2,3]) draw() print('continue computation') # at the end call show to ensure window won't close. show() Using interactive mode: from matplotlib.pyplot import plot, ion, show ion() # enables interactive mode plot([1,2,3]) # result shows immediatelly (implicit draw()) print('continue computation') # at the end call show to ensure window won't close. show() edited Jul 29, 2020 at 11:07 phoenix 5,36833 gold badges3232 silver badges3939 bronze badges answered Jan 19, 2009 at 16:52 nosklo 08********08 #] 22Mar2022 finish python typeout of code OK inital typout with errors - how do I load the [module, script]? +-----+ https://realpython.com/run-python-scripts/ How to Run Your Python Scripts by Leodanis Pozo Ramos 28 Comments Running Modules With the -m Option Python offers a series of command-line options that you can use according to your needs. For example, if you want to run a Python module, you can use the command python -m . The -m option searches sys.path for the module name and runs its content as __main__: $ python3 -m hello Hello World! Taking Advantage of import When you import a module, what really happens is that you load its contents for later access and use. The interesting thing about this process is that import runs the code as its final step. When the module contains only classes, functions, variables, and constants definitions, you probably won’t be aware that the code was actually run, but when the module includes calls to functions, methods, or other statements that generate visible results, then you’ll witness its execution. >> try $ python >>> import "$d_Python"'RKHS Roger Dev- Reproducing Kernel Hilbert Space.py' File "", line 1 import "$d_Python"'RKHS Roger Dev- Reproducing Kernel Hilbert Space.py' ^ SyntaxError: invalid syntax >>> import '/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS Roger Dev- Reproducing Kernel Hilbert Space.py' File "", line 1 import '/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS Roger Dev- Reproducing Kernel Hilbert Space.py' ^ SyntaxError: invalid syntax +-----+ https://stackoverflow.com/questions/9123517/how-do-you-import-a-file-in-python-with-spaces-in-the-name How do you import a file in python with spaces in the name? Asked 10 years, 1 month ago +--+ You should take the spaces out of the filename. Because the filename is used as the identifier for imported modules (i.e. foo.py will be imported as foo) and Python identifiers can't have spaces, this isn't supported by the import statement. If you really need to do this for some reason, you can use the __import__ function: foo_bar = __import__("foo bar") This will import foo bar.py as foo_bar. This behaves a little bit different than the import statement and you should avoid it. answered Feb 3, 2012 at 4:08 Jerem +--+ You can also use importlib.import_module function, which is a wrapper around __import__. foo_bar_mod = importlib.import_module("foo bar") or foo_bar_mod = importlib.import_module("path.to.foo bar") More info: https://docs.python.org/3/library/importlib.html edited Apr 23, 2020 at 15:45 gerrit 20.1k1414 gold badges7979 silver badges152152 bronze badges answered Dec 8, 2016 at 22:41 TBirkulosis +--+ Just to add to Banks' answer, if you are importing another file that you haven't saved in one of the directories Python checks for importing directories, you need to add the directory to your path with import sys sys.path.append("absolute/filepath/of/parent/directory/of/foo/bar") before calling foo_bar = __import__("foo bar") or foo_bar = importlib.import_module("foo bar") This is something you don't have to do if you were importing it with import , where Python will check the current directory for the module. If you are importing a module from the same directory, for example, use import os,sys sys.path.append(os.path.dirname(os.path.abspath(__file__))) foo_bar = __import__('foo_bar') Hope this saves someone else trying to import their own weirdly named file or a Python file they downloaded manually some time :) answered Sep 15, 2019 at 9:12 Anonymous Howell: This is the DUMBEST connvention I've seen in a long time!! There to FORCE Python progrs to have one class per file! Ridiculous... +-----+ Retry imports >>> import '/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS_Roger_Dev.py' File "", line 1 import '/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS_Roger_Dev.py' ^ SyntaxError: invalid syntax >>> import 'RKHS_Roger_Dev.py' File "", line 1 import 'RKHS_Roger_Dev.py' ^ SyntaxError: invalid syntax >>> import 'RKHS_Roger_Dev' File "", line 1 import 'RKHS_Roger_Dev' ^ SyntaxError: invalid syntax >>> import "RKHS_Roger_Dev"' File "", line 1 import 'RKHS_Roger_Dev' ^ SyntaxError: invalid syntax >>> import RKHS_Roger_Dev Traceback (most recent call last): File "", line 1, in File "RKHS_Roger_Dev.py", line 12 if type(dataPoints) == type([]) and 0; ^ SyntaxError: invalid syntax >> YAY!! Bingo, what a joke. Will go back to discover how to load any script, anywhere >>> import '/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS_Roger_Dev' File "", line 1 import '/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS_Roger_Dev' ^ SyntaxError: invalid syntax try syntx of : foo_bar_mod = importlib.import_module("path.to.foo bar") >>> RKHS_Roger_Dev = importlib.import_module('/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS_Roger_Dev') Traceback (most recent call last): File "", line 1, in NameError: name 'importlib' is not defined >>> __import__ '/media/bill/Dell2/Website - raw/Software programming & code/Python/RKHS_Roger_Dev' I give up for now...just go to subDir : $ cd '/media/bill/Dell2/Website - raw/Software programming & code/Python/' /media/bill/Dell2/Website - raw/Software programming & code/Python $ python Python 2.7.13 (default, Feb 6 2022, 20:16:18) [GCC 6.3.0 20170516] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import RKHS_Roger_Dev >> OK worked >>> import RKHS_Roger_Dev_example_test3 22Mar2022 16:8 works except plot no curves 08********08 #] 12Mar2022 type out code from images Reproducing Kernel Hilbert Spaces (RKHS) "$d_PROJECTS"'2022 WCCI Padua, Italy/reviews - mine/3806 r A Relative Spike-Timing Approach to Kernel-Based Decoding Demonstrated for Insect Flight Experiments.txt In [25], the author proposed a reproducingkernel Hilbert space (RKHS) framework that uses an instantaneous kernel to determine similarities between single spike trains directly. [25] A. R. Paiva, I. Park, and J. C. Principe, “A reproducing kernel hilbertspace framework for spike train signal processing,” Neural Computation,vol. 21, no. 2, pp. 424–449, 2009. [26] I. M. Park, S. Seth, A. R. Paiva, L. Li, and J. C. Principe, “Kernel methods on spike train space for neuroscience: a tutorial,” IEEE Signal Processing Magazine, vol. 30, no. 4, pp. 149–160, 2013. +-----+ Howell - fantastic aid!!!! : https://hpccsystems.com/blog/reproducing-RKHS Reproducing Kernel Hilbert Space (RKHS) -- A primer for non-mathematicians Roger Dev on 06/17/2021 Mathematically, RKHSs are a specific type of topological space with some very handy and surprising characteristics. It is almost impossible to imagine the shape of most RKHS spaces. They are typically very non-linear, and high to infinite dimensional. But here are some of the practical implications. It is a space of functions. Each point in space is a particular function. Function spaces tend to be infinite dimensional, so not only are there an infinite set of functions there, but the vector that identifies a given function can be of arbitrary (theoretically even infinite) length. Your sample data is typically used as the index into the function space. Index entries are real-valued. The functions are organized locally such that if the indices are similar, the resulting functions will be similar. The functions are smooth and continuous. Each function takes a single real-valued parameter, and returns a single real value. Linear functions in RKHS provide useful non-linear results. This is the unique characteristic of RKHS that makes it so useful. RKHSs are identified by a unique "kernel function". Each kernel induces exactly one RKHS, and each RKHS has a unique kernel function. One common use of RKHS is to find a function that describes a given variable. The data (samples of that variable) is given as the index into the function, and the function returned will estimate the probability density of the data's generating function. It is guaranteed to converge to the exact function that generated the data as the number of data points approaches infinity (with a few caveats*). Better than that, it will approach the correct distribution with every additional data point. Better still, every prediction that that function returns will be close to the actual value. "... Different kernels produce different types of functions, but some kernels are universal, meaning that they can emulate any continuous function to within isometric isomorphism, which is a fancy way of saying "exactly the same on the outside, though it may be different inside". ..." >> wow very well stated "... Now I'll share a secret about RKHS that you will seldom see exposed in any papers or articles. Implementing RKHS is embarrassingly easy. It may take hundreds of pages of math to describe RKHS, but only a few lines of code to make it work. The code sample at the bottom implements a flexible, optimized RKHS in under 30 lines of Python code. A hard coded RKHS could probably be done in 10 lines. It bothers me that it took several weeks of intense study to understand an algorithm that took me an hour to implement. That's what motivated me to write this article. ..." https://hpccsystems.com/ -> free, open source!!! tensorflow build-on too Appendix A -- Python Code Earlier in the article I talked about indexing into the infinite set of functions in the RKHS. In practice, your data is the index into your function space. But just to illustrate notions of function locality, and to provide a simple example, we will look at functions with various indexes that are close or far from each other. >> Howell: I will have to retype the pyton code... later? initially in QNial -> revert back to python /home/bill/Roger Dev - code[1,2].png /home/bill/Roger Dev 17Jun2021 Reproducing Kernel Hilbert Space (RKHS).ndf # enddoc