getattribute and setattr
'getattribute' and 'setattr' are two special methods (dunder methods) used to control access to a class's attributes. Here's a clear explanation of their role and differences:
getattribute(self, name)
Role: Called every time an attribute is accessed (read), even if the attribute doesn't exist
Default behavior: Searches for the attribute in the instance, then in the class, and then in the parent classes
usefulness
- Intercept access to an attribute to add logic (e.g., logging, validation)
- Raise an exception if the attribute is not allowed
Caution
If you override 'getattribute', you must call 'super().getattribute(name)' to avoid blocking access to internal attributes
setattr(self, name, value)
Role: Called whenever an attribute is modified (written)
Default behavior: Stores the value in the instance
usefulness
- Validate or modify the value before assigning it
- Prevent modification of certain attributes
- Synchronize attributes with other data
Caution
If you override 'setattr', you must use 'self.dict[name] = value' to avoid infinite recursion
example
class Temperature:
def __get__(self, obj, objtype):
print("desc __get__")
return obj._temperature
def __set__(self, obj, value):
print(f"desc __set__ {value}")
obj._temperature = value
class Maison:
def __init__(self, temperature):
self.temperature = temperature
def __getattribute__(self, attr):
print(f"__getattribute__ : {attr}")
return object.__getattribute__(self, attr)
def __setattr__(self, attr, value):
print(f"__setatt__ : {attr} - {value}")
return object.__setattr__(self, attr, value)
temperature = Temperature()
m = Maison(18)
__setatt__ : temperature - 18
desc __set__ 18
__setatt__ : _temperature - 18
m.temperature
__getattribute__ : temperature
desc __get__
__getattribute__ : _temperature
m.temperature = 22
__setatt__ : temperature - 22
desc __set__ 22
__setatt__ : _temperature - 22
m.x = 10
__setatt__ : x - 10
m.x
__getattribute__ : x
m.y
__getattribute__ : y
__getattribute__ : __dir__
__getattribute__ : __dict__
__getattribute__ : __class__
AttributeError: 'Maison' object has no attribute 'y'
getattr(self, name)
'getattr' is a special method (or dunder method) that is called when an attribute is not found in a class instance using the standard methods (getattribute, hasattr, etc.)
It allows you to dynamically manage access to missing attributes
usefulness
-
Creating attributes on the fly
-
Implementing default behaviors for undefined attributes
-
Proxies or wrappers: To redirect attribute calls to another object
-
Dynamic attributes: To generate values based on the attribute name
-
Compatibility with external APIs: To manage attributes that do not yet exist in a class
example
class Redirector:
def __init__(self, id):
self.id = id
def __repr__(self):
return f"Redirector2({self.id})"
def __getattr__(self, name):
def forwarder(arg):
return f"{self.id} -> {name}({arg})"
return forwarder
R = Redirector(2)
print(R)
print(R.bar(20))property
'@property' decorator allows you to define a method as a property of a class. This means you can access that method as if it were an attribute, without using parentheses to call a method.
- No parentheses: object.property (not object.property()).
- Convention: Use _<name> for the "private" attribute (e.g., _name).
- Validation: The setter can include checks.
- Compatibility: Works with inheritance and abstract classes.
example
class Circle:
def __init__(self, radius):
self._radius = radius
@property
def radius(self):
return self._radius
@radius.setter
def radius(self, value):
if value <= 0:
raise ValueError("The radius must be positive")
self._radius = value
@radius.deleter
def radius(self):
print("Deleting the radius")
del self._radius
c = Circle(5)
print(c.radius)
>>> 5
c.radius = 10
del c.radius
descriptor
A Python descriptor is an object that implements at least one of the following special methods:
__get__(self, obj, objtype=None)
__set__(self, obj, value)
__delete__(self, obj)
These methods allow you to customize access to, modification of, or deletion of an attribute of a class instance
usefulness
- Data validation: Check or transform values before assigning them
- Dynamic calculations: Generate a value on the fly (e.g., @property properties)
- Data sharing: Manage common attributes between multiple instances
examples
class DescriptorTemperature:
TEMPMAX = 20
def __get__(self, obj, objtype=None):
print('__get__')
return obj._temperature
def __set__(self, obj, value):
print('__set__')
if value > __class__.TEMPMAX:
raise ValueError(f"Temperature is too hot:{value}, max:{__class__.TEMPMAX} ")
obj._temperature = value
class Home:
def __init__(self, temperature):
self.temperature = temperature
temperature = DescriptorTemperature()
h = Home(18)
print(h.temperature)
h.temperature = 20
h.temperature = 30metaclass
A metaclass is a class that defines the behavior of other classes. By default, in Python, classes are instances of 'type', which is the default metaclass. However, you can create your own metaclasses to control class creation, add attributes, or modify class behavior.
usefulness
- Control class creation: Add attributes, validate properties, or modify the class before it is finalized
- Singleton: Ensure that a class has only one instance
- Automatic validation: Verify that certain methods or attributes exist in the classes that use them
- Automatic registration: Register classes in a registry (for example, for a plugin system)
new
- Role: 'new' is responsible for memory allocation and instance creation
- Signature: new(cls, name, bases, namespace)
- Return: It must return a new instance of the class (or another class if necessary)
usefulness
- To create singletons (a single instance of a class)
- To modify the type of the returned instance (for example, to return a subclass)
- To control object creation (e.g., validation before creation)
class LowerAttrType(type):
def __new__(cls, name, bases, namespace, **kwds):
namespace = {(n.lower() if not n.startswith('__') else n): obj for n, obj in namespace.items()}
bases = (BaseOfAll,)
return type.__new__(cls, name, bases, namespace, **kwds)
class BaseOfAll:
def common_func(self):
return f"in common_func"
class C(metaclass=LowerAttrType):
def fUnc_bAd_CAP(selfself):
return f"in fUnc_bAd_CAP"
c = C()
print('func_bad_cap', 'func_bad_cap' in c.__dir__())
>>> True
print('fUnc_bAd_CAP', 'fUnc_bAd_CAP' in c.__dir__())
>>> False
print('c.common_func', c.common_func())
>>> c.common_func in common_func
print('c.func_bad_cap', c.func_bad_cap())
>>> c.func_bad_cap in fUnc_bAd_CAPIn Python, a closure is a function that captures local variables from its lexical environment, even when the function is called outside of that environment.
The closure retains a reference to the parent function's variables, not their values at any given time
If the parent variable is modified after the closure is created, the closure will see the new value
def add_n(n):
def f(x):
return x + n
return f
add_10 = add_n(10)
print(add_10(100))
>>> 110
print('closure len', add_10.__closure__.__len__())
>>> 1
print('closure content', add_10.__closure__[0].cell_contents)
>>> 10
usefulness
Decorators: Closures are often used to create decorators
Function factories: Create functions with predefined behaviors
Context memory: Preserve state between function calls
modified closure value
nonlocal
def counter_call(f):
counter = 0
def wrapper(*args, **kargs):
nonlocal counter # set counter is non local attribute but free variable
counter += 1
r = f(*args, **kargs)
print(f"{f.__name__} called {counter} times")
return r
return wrapper
@counter_call
def poly(n):
return sum(i**5 for i in range(n))
function attribute
def counter_call(f):
def wrapper(*args, **kargs):
wrapper.counter += 1
r = f(*args, **kargs)
print(f"{f.__name__}() called {wrapper.counter} times")
return r
wrapper.counter = 0 # evaluate before wrapper calling
return wrapper
@counter_call
def poly(n):
return sum(i**5 for i in range(n))
r = poly(10)
>>> poly() called 1 times
print(poly(10))
>>> poly() called 2 times
print(poly.counter) # poly = wrapper(poly) et wrapper has attribute counter
>>> 2try except
https://docs.python.org/3/tutorial/errors.html#handling-exceptions
syntax
try:
....
# Catch only Error_name
except Error_class as e:
...
except Error_class2 as e:
...
else
...
# Execute always instructions at the end even after a return
finally
Catch all errors
try:
...
except:
...
...
example
def return_with_finally(number):
try:
return 1/number
except ValueError as e:
print(f"e.args, {e.args}")
return("zero-divide")
except ZeroDivisionError as e:
print(f"OOPS, {type(e)}, {e}")
return("zero-divide")
except TypeError as e:
# interrupt treatment with the error
raise
finally:
print("always executed even after a return in 'except'")
raise
http://www.python.50webs.com/CHAP_8/chap_8.html
Best Practices
- Use specific exceptions rather than generic ones
- Provide clear error messages
- Avoid throwing exceptions for normal control flow cases (prefer explicit returns).
launch a specific Error Class with text
raise ErrorClass("personalized message")
main class errors
AssertionError # Assertion (assert) failed
AttributeError # Attribute or method not found for an object
FileNotFoundError # File not found during an input/output operation
ImportError # Module or function not found during an import
IndentationError # Indentation problem (incorrect space or tab)
IndexError # Accessing an index outside the bounds of a list or string
KeyError # Key not found in a dictionary
MemoryError # The program has exhausted available memory
ModuleNotFoundError # Python module not found. Keyboard Interrupt # Manually interrupts the NameError # Undefined variable or function
NotImplementedError # Unimplemented method or function
OSError # Error related to the operating system (e.g., permissions problem)
RecursionError # Too many recursions (exceeds the depth limit)
RuntimeError # Generic error not covered by other exceptions
StopIteration # No more elements to iterate over (raised by the iterators)
SyntaxError # Syntax error in the code (missing parentheses, misspelled keyword, etc.)
TypeError # Operation or function applied to an inappropriate type (e.g., adding an integer to a string)
ValueError # Inappropriate value for an operation (e.g., converting a non-numeric string to an integer)
ZeroDivisionError # Division by zero
builtins class errors
ArithmeticError # Base class for arithmetic errors
AssertionError # Assertion failed
AttributeError # Attribute not found
BlockingIOError # I/O operation would block
BrokenPipeError # Broken pipe
BufferError # Buffer error
ChildProcessError # Child process error
ConnectionAbortedError # Connection aborted
ConnectionError # Connection error
ConnectionRefusedError # Connection refused
ConnectionResetError # Connection reset
EOFError # Read beyond end of file
EnvironmentError # Base class for I/O related errors
FileExistsError # File already exists
FileNotFoundError # File not found
FloatingPointError # Floating-point operation failed
IOError # Base class for I/O related errors
ImportError # Import can't find module, or can't find name in module
IndentationError # Improper indentation
IndexError # Sequence index out of range
InterruptedError # Interrupted by signal
IsADirectoryError # Operation doesn't work on directories
KeyError # Mapping key not found
LookupError # Base class for lookup errors
MemoryError # Out of memory
ModuleNotFoundError # Module not found
NameError # Name not found globally
NotADirectoryError # Operation only works on directories
NotImplementedError # Method or function hasn't been implemented yet
OSError # Base class for I/O related errors
OverflowError # Result too large to be represented
PermissionError # Not enough permissions
ProcessLookupError # Process not found
PythonFinalizationError # Operation blocked during Python finalization
RecursionError # Recursion limit exceeded
ReferenceError # Weak ref proxy used after referent went away
RuntimeError # Unspecified run-time error
SyntaxError # Invalid syntax
SystemError # Internal error in the Python interpreter
TabError # Improper mixture of spaces and tabs
TimeoutError # Timeout expired
TypeError # Inappropriate argument type
UnboundLocalError # Local name referenced but not bound to a value
UnicodeDecodeError # Unicode decoding error
UnicodeEncodeError # Unicode encoding error
UnicodeError # Unicode related error
UnicodeTranslateError # Unicode translation error
ValueError # Inappropriate argument value (of correct type)
ZeroDivisionError # Second argument to a division or modulo operation was zero
customize ErrorClass
For module-specific errors, use a prefix: 'class MyModuleError(Exception): ...'
For critical errors, inherit from 'RuntimeError' or 'ValueError'
Inheritance: The class inherits from Exception to be usable as a standard error.
init
Arguments:
- message: Error message (required).
- code: Error code (optional, defaults to 500).
- details: Dictionary for additional information (optional).
str
Displays the error in a readable format.
class CustomError(Exception):
'''Class for a custom error with arguments'''
def __init__(self, message: str, code: int = 500, details: dict = None):
self.message = message
self.code = code
self.details = details or {}
super().__init__(self.message)
def __str__(self):
return f"{self.message} (code: {self.code}, details: {self.details})"
example
try:
raise CustomError("An error has occurred", 404, {"file": "data.txt"})
except CustomError as e:
print(e) # print: "An error has occurred (code: 404, details: {'file': 'data.txt'})"
print(e.code) # print: 404
print(e.details) # print: {'file': 'data.txt'})"
assert
If assert condition return False a AssertionError exception is always launched
- Assertions are primarily used during development.
- It is imperative to avoid side effects in assertions because,
when they are removed, the effect is also removed. - Assertions can be removed from the bytecode if the -O option is called during compilation.
assert condition, "error_message"
# assert is equal to:
if __debug__ :
if not <test> :
raise AssertionError(<values>)static method
if inherited method as redefined, decorator @staticmethod must be added
usefulness
- For utility functions related to the class (e.g., calculations, conversions).
- When the method does not need to access the class or instance's data.
- To avoid cluttering the global namespace with independent functions.
examples
class Sentence:
_call_counter = 0
@staticmethod
def call_counter():
return Sentence._call_counter
def __init__(self, sentence = ''):
Sentence._call_counter += 1
self.sentence = sentence
def __repr__(self):
return f"{__class__} called {self.call_counter()}"
class SentenceLower(Sentence):
_call_counter = 0
@staticmethod
def call_counter():
return SentenceLower._call_counter
def __init__(self, sentence = ''):
SentenceLower._call_counter += 1
self.sentence = sentence.lower()
s1 = Sentence("ma phrase s1")
s2 = SentenceLower("ma phrase s2")
s3 = SentenceLower("ma phrase s3")
print('s1', s1)
>>> s1 <class '__main__.Sentence'> called 1
print('s2', s2)
>>> s2 <class '__main__.Sentence'> called 2
print(s1.call_counter(), Sentence.call_counter())
>>> 1, 1
print(s2.call_counter(), SentenceLower.call_counter())
>>> 2, 2
class method
if inherited method as redefined, decorator @classmethod must be added
usefulness
- Create alternative methods for constructing instances (like factory methods)
- Access or modify class attributes
- Do not depend on a specific instance
class Sentence:
_call_counter = 0
@classmethod
def call_counter(cls):
return cls._call_counter
def __init__(self, sentence = ''):
__class__._call_counter += 1
self.sentence = sentence
def __repr__(self):
return f"{self.__class__} called {self.call_counter()}"
class SentenceLower(Sentence):
_call_counter = 0
def __init__(self, sentence = ''):
__class__._call_counter += 1
self.sentence = sentencehttps://wiki.python.org/moin/PythonDecoratorLibrary
function decorator
A decorator in Python is a function that allows you to modify or extend the behavior of another function or method without altering its source code. It is a powerful tool for adding reusable functionality (such as logging, time measurement, validation, etc.) in a clean and modular way.
@decorator
def my_function(): pass
# is equivalent to
def my_function(): pass
my_function = decorator(my_function)
usefulness
- Custom logging
- Argument validation
- Caching (Memoization)
- Timing
decorator as a class
class without arguments
class CallCounter:
def __init__(self, f):
self.call_counter = 0
self.f = f
def __call__(self, *args, **kargs):
self.call_counter += 1
print(f"{self.f.__name__} : {self.call_counter} call(s)")
return self.f(*args, **kargs)
@CallCounter
def accumulate(*args):
if args and len(args) == 1 and hasattr(args, '__iter__'):
args = args[0]
sum_items = 0
return sum(sum_items := sum_items + i for i in args)
print(accumulate(1,2,3,4,5,6,7,8,9))
>>> accumulate : 1 call(s)
>>> accumulate(10) 165
print(accumulate(range(10)))
>>> accumulate : 2 call(s)
>>> accumulate(10) 165
print(accumulate(range(100)))
>>> accumulate : 3 call(s)
>>> accumulate(10) 166650
class with arguments
class CallCounter:
def __init__(self, step = 1):
self.step = step
self.call_counter = 0
def __call__(self, f):
def wrapper(*args, **kargs):
self.call_counter += self.step
print(f"{f.__name__}: {self.call_counter} call(s)")
return f(*args, **kargs)
return wrapper
@CallCounter(step = 2)
def accumulate(*args):
if args and len(args) == 1 and hasattr(args, '__iter__'):
args = args[0]
sum_items = 0
return sum(sum_items := sum_items + i for i in args)
print(accumulate(1,2,3,4,5,6,7,8,9))
print(accumulate(range(10)))
print(accumulate(range(20)))
decorator as a function
function without arguments
a timer
def mytimer(function):
from datetime import datetime
def wrapper(*args, **kwargs):
d1 = datetime.now()
result = function(*args, **kwargs)
d2 = datetime.now()
print(*args, '->', result)
print('result', result)
print(d2-d1)
return result
return wrapper
@mytimer
def powerful_digit_sum(limit):
max_sum = (2, 2, 1)
for a in range(2, limit):
for b in range(1, limit):
sum_digits = sum(int(i) for i in str(a**b))
max_sum = (sum_digits, a, b) if sum_digits > max_sum[0] else max_sum
return max_sum
limit = 100
powerful_digit_sum(limit)
memoize or cache
def memoize(f):
'''decorator to keep data in cache'''
from functools import wraps
# preserves the metadata of f and not of wrapper. See below functools
@wraps(f)
def wrapper(*args):
try:
return wrapper.cache[args]
except TypeError:
return f(*args)
except KeyError:
result = f(*args)
wrapper.cache[args] = result
return result
# instantiated in module interpretation, before fibo_cache() calling.
# define after function definition
# wrapper is returned to fibo_cache: fibo_cache = memoize(fibo_cache)
wrapper.cache = {}
return wrapper
# @functools.cache # functools.cache is twice as fast. See below
@memoize
def fibo_cache(n):
'''One super-slow fibonacci'''
return n if n <= 1 else fibo_cache(n-1) + fibo_cache(n-2)
print(fibo_cache(100))
>>> 354224848179261915075
print(len(fibo_cache.cache))
>>> 101
print(fibo_cache.__doc__) # thanks to functools.wrap
>>> One super-slow fibonacci
function with arguments
specific cache
def cache_with_size(max_size):
def decorator(f):
cache = {}
def wrapper(*args):
if args in cache:
print("Result retrieved from cache")
return cache[args]
result = f(*args)
if len(cache) >= max_size:
# Remove the first inserted element (FIFO)
cache.pop(next(iter(cache)))
cache[args] = result
return result
return wrapper
return decorator
@cache_with_size(max_size=2)
def expensive_function(x):
print(f"Calculation for {x}...")
return x * x
# Appel
print(expensive_function(2)) # Calculation for 2... → 4
print(expensive_function(3)) # Calculation for 3... → 9
print(expensive_function(2)) # Result retrieved from cache → 4
def memoize_expire(timeout):
'''decorator to keep data in cache with an expiration delay'''
def memoize(f):
import time
from functools import wraps
# preserves the metadata of f and not of wrapper. See below functools
@wraps(f)
def wrapper(*args):
try:
value, timestamp = wrapper.cache[args]
if (time.time() - timestamp) <= timeout:
return value
else:
raise KeyError
except TypeError:
return f(*args)
except KeyError:
value = f(*args)
wrapper.cache[args] = value, time.time()
return value
# instantiated in module interpretation, before fibo_cache() calling.
# define after function definition
# wrapper is returned to fibo_cache: fibo_cache = memoize(fibo_cache)
wrapper.cache = {}
return wrapper
return memoize
# @functools.cache # functools.cache is twice as fast. See below
@memoize_expire(600)
def fibo_cache(n):
'''One super-slow fibonacci'''
return n if n <= 1 else fibo_cache(n-1) + fibo_cache(n-2)
print(fibo_cache(100))
class decorator
A class decorator in Python is a function that allows you to modify or extend the behavior of a class without directly modifying its code. It applies to the class itself, unlike method decorators which apply to methods.
@decorator
class MyClass: pass
# is equivalent to
class MyClass: pass
MyClass = decorator(MyClass)
@decorator(arg)
class MyClass: pass
# is equivalent to
class MyClass: pass
MyClass = (decorator(arg))(MyClass)
usefulness
- Add metadata (version, author, etc.)
- Register classes in a plugin system
- Modify method behavior (logging, validation, etc.)
- Apply mixins dynamically
- Check or validate the class before use
decorator as a function
function without arguments
def has_add(cls):
if not hasattr(cls, '__add__'):
raise AttributeError(f"{cls.__name__} missing attribute: __add__")
return cls
@has_add
class Foo:
'''test class for class decorator'''
def __str__(self):
return f"repr {__class__} !"
f = Foo()
print(f)
function with arguments
def add_field(argdec):
def decorator(cls):
setattr(cls, argdec, 'default')
return cls
return decorator
@add_field('name')
@add_field('address')
class Foo:
'''test class for class decorator'''
def __repr__(self):
return f"{__name__}: name: {self.name} address: {self.address}"
f = Foo()
print(f)
print(f.__doc__)
functools
@wraps
'functools.wraps' is a decorator built into Python's 'functools' module. It is used to preserve the metadata of an original function when it is decorated by another function (a wrapper).
import functools
@functools.cache
def my_function():
pass
@cache
import functools
@functools.cache
def my_function():
pass
@lru_cache
import functools
@functools.lru_cache(maxsize=128, typed=False)
def my_function():
pass
decorators' collection
install
pipx
sudo pacman -S python-pipx # archlinux, manjaro, ...
sudo apt-get install pipx # ubuntu, debian, ...
pipx install pipenv
pip
sudo pacman -S python-pipx # archlinux, manjaro, ...
sudo apt-get install python3-pip # ubuntu, debian, ...
pip install pipenv
use
mkdir my_project
cd my_project
pipenv --version # show program's version number and exit
pipenv --python <version> # install Python version if its available (use pyenv)
pipenv install requests # required packages
pipenv install numpy matplot ... # install pip packages
pipenv run pip list # List all pip installed packages
pipenv graph # Display currently-installed dependency graph information
pipenv uninstall <package> # remove pip package from virtualenv
pipenv shell # Spawn a shell within the virtualenv
pipenv run python script.py # Spawn a command installed into the virtualenv
pipenv --where # Output project home information.
pipenv --venv # Output virtualenv information.
pipenv --py # Output Python interpreter information
pipenv requirements # Generate a requirements.txt from Pipfile.lock
Pipfile : Liste des dépendances et des packages en développement
Pipfile.lock : Version exacte des dépendances installées (pour la reproductibilité)
pipenv remove # Remove the virtualenv for the current project
cd my_project && rm Pipfile Pipfile.lock # delete files from project directory
pipx install --user --upgrade pipenv # update the packages regularly
eclipse
packages
pipenv install matplot modtools numpy pandas setuptools sympy
configuration
- select Windows/Preferences/Pydev/Interpreters/Python interpreter
- New/New with pipenv
- select the path given by 'pipenv --where' for the first field: 'Project location'
- click directly on 'create Pipenv interpreter'
install
sudo pacman -S tk pyenv # archlinux, manjaro, ...
sudo apt-get install pyenv python3-tk # ubuntu, debian, ...
use
pyenv versions # list all installed Python version(s)
pyenv install --list # list all available Python versions
pyenv install <version> # install a specific Python version
pyenv version # Show the current Python version(s) and its origin
pyenv version-file # Detect the file that sets the current pyenv version
pyenv version-name # Show the current Python version
pyenv version-origin # Explain how the current Python version is set
pyenv uninstall <version> # Uninstall Python version
pyenv global # Set or show the global Python version(s)
pyenv local # Set or show the local application-specific Python version(s)
pyenv exec # Run an executable with the selected Python version
pyenv shims # List existing pyenv shims
pyenv whence # List all Python versions that contain the given executable
pyenv which # Display the full path to an executableInstall
https://virtualenv.pypa.io/en/latest/installation.html
manjaro
sudo pacman -S --needed python python-pipx
pipx install virtualenv
Use
https://virtualenv.pypa.io/en/latest/cli_interface.html#creator
virtualenv <path> # create the virtual environment in <path>
virtualenv --creator <path> # create the virtual environment in <path> with specific creator like : cpython3-mac-brew, cpython3-mac-framework, cpython3-posix, cpython3-win, graalpy-posix, graalpy-win, pypy3-posix, pypy3-win, venv
<path>/bin/activate # activate the virtual environment
deactivate # deactivate the virtual environment
which python3 # to view the referred python pah ReST
https://rest-sphinx-memo.readthedocs.io/en/latest/
Syntax
code
module,class,method
:var <TYPE> <NOM>: <Description de la variabl>
:var <NOM>: <Description de la variabl>
:param <TYPE> <NOM>: <Description du paramètre>.
:param <NOM>: <Description d'un autre paramètre. Ici on ne précise pas le type, c'est optionnel>.
:returns: <Description de ce qui est retourné (si la fonction retourne quelque chose)>.
:rtype: <Type de ce qui est retourné>
:raises <Exception>: <Description de l'exception>.
:myindex: (int): deep
reference
`link vers ce siteb <https://code.ambau.ovh/>`_
:class:`mymodule.MyClass` About my class.
:meth:`mymodule.MyClass.myfunction` And my cool function
format
*italique*
**gras**
`code inline``
directives
https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
.. NOTE:: Ceci est une note.
.. WARNING:: Ceci est un avertissement !
.. IMPORTANT:: Ceci est important
.. TODO:: Liste de choses à faire
.. IMAGE:: files/graph1.png # from source path & html path
title
Titre principal
===============
Titre de niveau 2
-----------------
Titre de niveau 3
~~~~~~~~~~~~~~~~~
Un autre titre de niveau 2
--------------------------
list
* Ceci est une liste
* un autre élément
* Une sous-liste
* notez bien le saut de ligne avec la liste principale,
ça ne marchera pas si vous l'oubliez !
* un dernier élément
1. Un
2. Deux
3. Troishttps://rest-sphinx-memo.readthedocs.io/en/latest/ReST.html
#. Un
#. Deux
#. Trois
table
+-----------+-----------+-----------+
| Heading 1 | Heading 2 | Heading 3 |
+===========+===========+===========+
| Hello | World | |
+-----------+-----------+-----------+
| foo | |
+-----------+ bar |
| baz | |
+-----------+-----------------------+
rst files
index.rst
TITLE 1
=======
.. toctree::
:maxdepth: 2
:caption: Contents:
./AMatrix.rst
name.rst
include
include:: ../README.rst
module
.. module:: parrot
:platform: Unix, Windows
:synopsis: Analyze and reanimate dead parrots.
.. moduleauthor:: Eric Cleese <eric@python.invalid>
.. moduleauthor:: John Idle <john@python.invalid>
auto
.. automodule:: package.module # or just .. automodule:: module
.. autoclass:: classname
.. autofunction:: functionname
.. automethod:: myproject.foomodule.Foo.__contains__
options
:show-inheritance:
:members:
:private-members:
:undoc-members:
:special-members:
:exclude-members: __doc__
examples
.. automodule:: modulename
:members:
:undoc-members:
:exclude-members: __dict__,__weakref__
Installation
# manjaro
yay -S python-sphinx python-sphinx_rtd_theme
Use
https://deusyss.developpez.com/tutoriels/Python/SphinxDoc/
https://blog.flozz.fr/2020/09/07/introduction-a-sphinx-un-outil-de-documentation-puissant/
https://romanvm.pythonanywhere.com/post/autodocumenting-your-python-code-sphinx-part-i-5/
https://samnicholls.net/2016/06/15/how-to-sphinx-readthedocs/
sphinx-quickstart
Go to your project path
cd $path2myproject
mkdir docs && cd docs
sphinx-quickstart
export PYTHONPATH=$path2myproject
make html
sphinx-apidoc
sphinx-apidoc [OPTIONS] -o <OUTPUT_PATH> <MODULE_PATH> [EXCLUDE_PATTERN …]
Configuration
cd $path2myproject
file="source/conf.py"
sed -i "s|.*\(import os\)$|\1|" "${file}"
sed -i "s|.*\(import sys\)$|\1|" "${file}"
sed -i "s|.*sys.path.insert.*|sys.path.insert(0, os.path.abspath('../../'))|" "${file}"
sed -i "s|^\(extensions = [.*\)$|\1\n\t'sphinx.ext.autodoc',|" "${file}"
sed -i "s|^\(extensions = [.*\)$|\1\n\t'sphinx_rtd_theme',|" "${file}"
sed -i "s|^\(html_theme = .*\)$|\1\nhtml_theme = 'sphinx_rtd_theme'|" "${file}"
make html
conf.py
path
import os
import sys
# sys.path.insert(0, os.path.abspath('.'))
sys.path.insert(0, os.path.abspath('../../'))
extensions
extensions = [
'sphinx.ext.autodoc',
'sphinx_rtd_theme',
'sphinx.ext.todo',
'sphinx.ext.autosummary',
'sphinx_autopackagesummary',
]
[extensions]
todo_include_todos=True
#autosummary_generate = True
autodoc_member_order = 'bysource' # alphabetical (alphabetical), groupwise(member type), bysource (source order)install
conf
sudo sh -c "echo '\n# sublime-text hack\n127.0.0.1\tsublimetext.com\n127.0.0.1\twww.sublimetext.com\n127.0.0.1\tlicense.sublimehq.com' >> /etc/hosts"
apt install -y iptables-persistent
ips="45.55.255.55"
for ip in ${ips}; do sudo iptables -A OUTPUT -d ${ip} -j DROP; done
path="/etc/iptables"
[ -d "${path}" ] || sudo mkdir "${path}"
sudo iptables-save -f /etc/iptables/rules.v4
build 3211
----- BEGIN LICENSE -----
Member J2TeaM
Single User License
EA7E-1011316
D7DA350E 1B8B0760 972F8B60 F3E64036
B9B4E234 F356F38F 0AD1E3B7 0E9C5FAD
FA0A2ABE 25F65BD8 D51458E5 3923CE80
87428428 79079A01 AA69F319 A1AF29A4
A684C2DC 0B1583D4 19CBD290 217618CD
5653E0A0 BACE3948 BB2EE45E 422D2C87
DD9AF44B 99C49590 D2DBDEE1 75860FD2
8C8BB2AD B2ECE5A4 EFC08AF2 25A9B864
------ END LICENSE ------
build 3176
—– BEGIN LICENSE —–
sgbteam
Single User License
EA7E-1153259
8891CBB9 F1513E4F 1A3405C1 A865D53F
115F202E 7B91AB2D 0D2A40ED 352B269B
76E84F0B CD69BFC7 59F2DFEF E267328F
215652A3 E88F9D8F 4C38E3BA 5B2DAAE4
969624E7 DC9CD4D5 717FB40C 1B9738CF
20B3C4F1 E917B5B3 87C38D9C ACCE7DD8
5F7EF854 86B9743C FADC04AA FB0DA5C0
F913BE58 42FEA319 F954EFDD AE881E0B
—— END LICENSE ——
packagecontrol
https://packagecontrol.io # packages info + shortcuts
COLOR SCHEME
package control / Install Package / PackageResourceViewer
package control / PackageResourceViewer / Open Resource / select scheme
package control / PackageResourceViewer / Theme - Default / Adaptive.sublime-theme
package control / PackageResourceViewer / Theme - Default / adaptive / Adaptive.sublime-theme
SIZE
/home/$USER/.config/sublime-text-3/Packages/Theme - Default/adaptive/Adaptive.sublime-theme
/home/$USER/.config/sublime-text-3/Packages/Theme - Default/Default.sublime-theme
PACKAGE
python
- Anaconda
Anaconda turns your Sublime Text 3 in a full featured Python development IDE - Python Debugger
Graphical Debugger for Sublime Text - Python 3
Python 3 and Cython language bundles for Sublime Text and TextMate - DocBlockr_Python
Sublime Text DocBlockr for python. Simplifies writing docstring comments in Python
TYPES
str
unicode
list
tuple
buffer
xrange
PRIORITY ORDER
x in s # True if an item of s is equal to x, else False
x not in s # False if an item of s is equal to x, else True
s + t # the concatenation of s and t
s * n , n * s # n shallow copies of s concatenated
s[i] # i'th item of s, origin 0
s[i:j] # slice of s from i to j
s[i:j:k] # slice of s from i to j with step k
len(s) # length of s
min(s) # smallest item of s
max(s) # largest item of s
METHOD
s.index(str) # return first position of str in s
s.count() # return then number of elements
SLICE
first included, second excluded (even with negative step)
s[start:end:step]
syntax
s[start:]
s[:end]
s[:] # duplicates element
s[start:-end] # from index1 to (index2 from last)
s[-start:-end] # from (index1 from last) to (index2 from last)
s[-start:end] # from (index1 from last) to index2 -> empty
s[-start:end:-step] # from (index1 from last) to (index2 from last)
s[::-1] # reverse
s[start:end:-2] # with start > end else return None # reverse a part with step
s[start;start] = [n] # insert element n in s at position start
s[start:end] = [] # unset elements between start and end (start included & end excluded)
examples
l = [0, 2, 4, 8, 16, 32, 64, 128]
l
l[-1:1:-2]
l[2:4] = [100, 200, 300, 400, 500]
lhttps://docs.python.org/3/reference/datamodel.html#with-statement-context-managers
Allow to execute codes define in object himself in enter & exit methods
Propagates an exception based on the return value of the exit method:
- False -> propagate
- True -> does not propagate
Difference between try/finally & with/as are about the place of codes executed in finally or enter/exit:
- for try/finally, codes are in codes arround object
- for with/as, codes are in embedded in object
syntax
with expression as name:
...
catch specific exception in Timer.__exit__()
def __exit__(self, exc_type, exc_value, traceback):
example
with as
import time
class Timer:
def __enter__(self):
print("Entering Timer")
self.start = time.time()
return self
def __exit__(self, *args):
print(f"execution time {time.time() - self.start}")
# propagate exception
return False
def __str__(self):
return f"delay {time.time() - self.start}s"
with Timer() as t:
sum(x for x in range(10_000_000))
1/0
print(t)
sum(x**2 for x in range(10_000_000))
or
import time
try:
with Timer():
time.sleep(0.5)
1/0
except Exception as exc:
# on va bien recevoir cette exception
print(f"OOPS -> {type(exc)}")
try finally
import time
try:
start = time.time()
print(sum(x for x in range(10_000_000)))
1/0
print(f"delay {time.time() - start}s")
print(sum(x**2 for x in range(5_000_000)))
except ZeroDivisionError as e:
print(f"OOPS, {type(e)}, {e}")
finally:
print(f"execution time {time.time() - start}")
catch specific exception
class Timer2:
def __enter__(self):
print("Entering Timer1")
self.start = time.time()
return self
def __exit__(self, exc_type, exc_value, traceback):
if exc_type is None:
# pas d'exception levée dans le corps du 'with'
print(f"Total duration {time.time()-self.start:2f}")
# dans ce cas la valeur de retour n'est pas utilisée
else:
if exc_type in (ZeroDivisionError,) :
print("on étouffe")
# on peut l'étouffer en retournant True
return True
else:
print(f"OOPS : on propage l'exception "
f"{exc_type} - {exc_value}")
# et pour ça il suffit... de ne rien faire du tout
# ce qui renverra None
with an other exception
try:
with Timer2():
time.sleep(0.5)
raise OSError()
except Exception as e:
# on va bien recevoir cette exception
print(f"OOPS -> {type(e)}")
decorator
example
from contextlib import contextmanager
@contextmanager
def compact_timer(message):
start = time.time()
yield
print(f"{message}: duration = {time.time() - start}")
with compact_timer("Squares sum"):
print(sum(x**2 for x in range(10**5)))https://docs.python.org/3/reference/datamodel.html#specialnames
https://docs.python.org/3/reference/datamodel.html#object.__new__
https://docs.python.org/3/reference/datamodel.html#object.__repr__
list of mains specials methods
Special methods (or dunder methods, for double underscore) are methods prefixed and suffixed with __ that define the behavior of objects for built-in operators, built-in functions, or specific protocols (such as iteration or comparison).
Initialization and Destruction
__init__(self, ...) # Object constructor
__del__(self) # Destructor (called when the object is deleted)
Representation and Display
__repr__(self) # "Official" representation of the object (used by repr())
__str__(self) # "Informal" representation (used by str() and print())
__format__(self, format_spec) # Behavior for format() and f-strings
__bytes__(self) # Behavior for bytes()
Accessing Attributes
__getattr__(self, name) # Called when an attribute is not found
__getattribute__(self, name) # Called for all attribute accesses
__setattr__(self, name, value) # Called when an attribute is assigned
__delattr__(self, name) # Called when an attribute is deleted
__dir__(self) # List of accessible attributes (used by dir())
Arithmetic and logical operators
Unary operators
__pos__(self) # +x
__neg__(self) # -x
__abs__(self) # abs(x)
__invert__(self) # ~x (binary inversion)
Binary operators
__add__(self, other) # x + y
__sub__(self, other) # x - y
__mul__(self, other) # x * y
__matmul__(self, other) # x @ y (matrix multiplication)
__truediv__(self, other) # x / y
__floordiv__(self, other) # x // y
__mod__(self, other) # x % y
__pow__(self, other) # x ** y
__lshift__(self, other) # x << y
__rshift__(self, other) # x >> y
__and__(self, other) # x & y
__or__(self, other) # x | y
__xor__(self, other) # x ^ y
Comparison operators
__eq__(self, other) # x == y
__ne__(self, other) # x != y
__lt__(self, other) # x < y
__le__(self, other) # x <= y
__gt__(self, other) # x > y
__ge__(self, other) # x >= y
Augmented assignment operators
__iadd__(self, other) # x += y
__isub__(self, other) # x -= y
__imul__(self, other) # x *= y
__imatmul__(self, other) # x @= y
__itruediv__(self, other) # x /= y
__ifloordiv__(self, other) # x //= y
__imod__(self, other) # x %= y
__ipow__(self, other) # x **= y
__ilshift__(self, other) # x <<= y
__irshift__(self, other) # x >>= y
__iand__(self, other) # x &= y
__ior__(self, other) # x |= y
__ixor__(self, other) # x ^= y
Container protocols
__len__(self) # len(x) (length).
__getitem__(self, key) # x[key] (access by index/key).
__setitem__(self, key, value) # x[key] = value.
__delitem__(self, key) # del x[key].
__contains__(self, item) # item in x (membership test).
__iter__(self) # Returns an iterator for iter(x).
__next__(self) # Next value for the iterator (used with __iter__)
__reversed__(self) # Returns a reversed iterator for reversed(x)
Context Protocols (Resource Management)
__enter__(self) # Entry into a with block
__exit__(self, exc_type, exc_val, exc_tb) # Exit from a with block
Serialization Protocols
__reduce__(self) # Used by pickle for serialization
__reduce_ex__(self, protocol) # Extended version of __reduce__
Copy Protocols
__copy__(self) # Behavior for copy.copy()
__deepcopy__(self, memo) # Behavior for copy.deepcopy()
Type Conversion Protocols
__int__(self) # Converts to an integer
__float__(self) # Converts to a float
__complex__(self) # Converts to a complex number
__index__(self) # Converts to an integer for indexing (used by lists, tuples, etc.)
__round__(self, n) # Behavior for round(x, n)
__trunc__(self) # Behavior for math.trunc(x)
Hash and Equality Protocols
__hash__(self) # Returns a hash for the object (used in dictionaries and sets)
__eq__(self, other) # Already mentioned in the comparison operators
Call and Instance Protocols
__call__(self, *args, **kwargs) # Allows calling an instance like a function # x()
__instancecheck__(self, instance) # Used by isinstance() (in a metaclass)
__subclasscheck__(self, subclass) # Used by issubclass() (in a metaclass)
Dynamic Attribute Management Protocols
__get__(self, instance, owner) # Behavior for descriptors (used in class properties and methods)
__set__(self, instance, value) # Behavior for writable descriptors
__delete__(self, instance) # Behavior for deleted descriptors
Module Management Protocols
__dir__(self) # List of accessible attributes (already mentioned)
__getattr__(self, name) # Already mentioned
Class Management Protocols
__prepare__(name, bases) # Used in a metaclass to create the class namespace
__new__(cls, *args, **kwargs) # Creates a new instance of the class (before __init__)
__init_subclass__(cls) # Called when a class is subclassed
Coroutine and Async Management Protocols
__await__(self) # Allows the use of an object in an await expression
__aiter__(self) # Returns an asynchronous iterator for aiter(x)
__anext__(self) # Next value for the asynchronous iterator
__aenter__(self) # Enters an async with block
__aexit__(self, exc_type, exc_val, exc_tb) # Exits an async with block
Array and Buffer Management Protocols
__array__(self) # Used by NumPy to convert an object to an array
__buffer__(self, flags) # Used for buffer protocol objects (such as MemoryView)
File and I/O Management Protocols
__fspath__(self) # Returns a file path (used by os.fspath())
Event Handling Protocols
__await__(self) # Already mentioned
examples
__call__
class PlusClosure:
def __init__(self, initial):
self.initial = initial
def __call__(self, *args):
return self.initial + sum(args)
plus2 = PlusClosure(2)
plus2(5)
__getattr__
class RPCProxy:
def __init__(self, url, login, password):
self.url = url
self.login = login
self.password = password
def __getattr__(self, function):
"""
Crée à la volée une méthode sur RPCProxy qui correspond
à la fonction distante 'function'
"""
def forwarder(*args):
print(f"Envoi à {self.url}...")
print(f"de la fonction {function} -- args= {args}")
return "retour de la fonction " + function
return forwarder
rpc_proxy = RPCProxy (url='http://cloud.provider.com/JSONAPI',
login='dupont',
password='***')
# cette partie du code, en tant qu'utilisateur de l'API, est supposée connaître les détails des arguments à passer et de comment utiliser les valeurs de retour
nodes_list = rpc_proxy.GetNodes (
[ ('phy_mem', '>=', '32G') ] )
# réserver un noeud
node_lease = rpc_proxy.BookNode (
{ 'id' : 1002, 'phy_mem' : '32G' } )
Point
class Point:
a = 0
b = 0
d = [a, b]
def __init__(self, x=0, y=0, len=0):
self.x = x
self.y = y
self.p = [x, y]
def __repr__(self):
return f"Pt[{self.x}, {self.y}]"
class Point_eh(Point):
def __eq__(self, other):
return self.x == other.x and self.y == other.y
def __hash__(self):
return (11 * self.x + self.y)
p1 = Point(0,1)
p2 = Point(0,1)
p3 = Point(0,1)
s = {p1, p2}
p1
s
# __eq__
p1 == p2
peh1 = Point_eh(0,1)
peh2 = Point_eh(0,1)
s = {peh1, peh2}
Matrix
class Matrix:
def __init__(self, *args):
"""
le constructeur accepte
(*) soit les 4 coefficients individuellement
(*) soit une liste - ou + généralement une séquence - des mêmes
"""
# on veut pouvoir créer l'objet à partir des 4 coefficients
# souvenez-vous qu'avec la forme *args, args est toujours un tuple
if len(args) == 4:
self.coefs = args
# ou bien d'une séquence de 4 coefficients
elif len(args) == 1:
self.coefs = tuple(*args)
def __repr__(self):
"l'affichage"
return "[" + ", ".join([str(c) for c in self.coefs]) + "]"
def __add__(self, other):
"""
l'addition de deux matrices retourne un nouvel objet
la possibilité de créer une matrice à partir
d'une liste rend ce code beaucoup plus facile à écrire
"""
return Matrix([a + b for a, b in zip(self.coefs, other.coefs)])
def __bool__(self):
"""
on considère que la matrice est non nulle
si un au moins de ses coefficients est non nul
"""
# ATTENTION le retour doit être un booléen
# ou à la rigueur 0 ou 1
for c in self.coefs:
if c:
return True
return False
Remarquez que les opérandes sont apparemment inversés dans le sens où pour evaluer 'reel * matrice'
On écrit une méthode qui prend en argument la matrice, puis le réel mais n'oubliez pas qu'on est en fait en train d'écrire une méthode sur la classe Matrix2
def multiplication_scalaire(self, alpha):
return Matrix2([alpha * coef for coef in self.coefs])
on ajoute la méthode spéciale rmul
Matrix2.__rmul__ = multiplication_scalaireclass
type
class C: pass
c = C()
C.__class__
>>> <class 'type'>
c.__class__
>>> <class '__main__.C'>
attributes
class C:
pass
C.__class__
>>> <class 'type'>
C.__name__
>>> C
dir(C)
>>> ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__firstlineno__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__static_attributes__', '__str__', '__subclasshook__', '__weakref__']
vars(C)
>>> {'__module__': '__main__', '__firstlineno__': 21, '__static_attributes__': (), '__dict__': <attribute '__dict__' of 'C' objects>, '__weakref__': <attribute '__weakref__' of 'C' objects>, '__doc__': None}
c = C()
c.__class__
>>> <class '__main__.C'>
C.__name__
>>> c
dir(c)
>>> ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__firstlineno__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__static_attributes__', '__str__', '__subclasshook__', '__weakref__']
vars(c)
>>> {}
scope
For class attributes, the resolution order follow the tree of inheritance
For variables use LEGB
- Local
- Enclosing
- Global
- Built-in
a = 1
class C():
a = 2
class B():
def f(self):
def g():
print('a in g =', a, '& C.a =', C.a)
g()
C.B().f()
>>> a in g = 1 & C.a = 2
a = 1
class C():
a = 2
class B():
def f(self):
a = 3
def g():
print('a in g =', a, '& C.a =', C.a)
g()
C.B().f()
>>> a in g = 3 & C.a = 2
inheritance
the declaration order of inherited classes is important pour mro()
class Myclass(parentclass1, parentClass2, ...)
super().method_name() can be used in place of Class_parent_name.method_name(self)
the declaration order of inherited classes is important pour mro()
encapsulation
_attribute: Convention to indicate that the attribute is "protected" (should not be accessed directly outside the class or its subclasses)
__attribute: Makes the attribute "private" (mangling name applied to avoid conflicts in subclasses)
bound method
Declaration type:
- implicit: if the girl class does not define the method at all
- redefined: if one rewrites the method entirely
- modified: if we rewrite the method in the class girl, but using the code of the mother class
class Fleur:
def implicite(self):
print('Fleur.implicite')
def redefinie(self):
print('Fleur.redéfinie')
def modifiee(self):
print('Fleur.modifiée')
class Rose(Fleur):
def redefinie(self):
print('Rose.redefinie')
def modifiee(self):
Fleur.modifiee(self)
print('Rose.modifiee apres Fleur')
# with use of super()
class Rose(Fleur):
def modifiee(self):
super().modifiee()
print('Rose.modifiee apres Fleur')
multiple inheritance
Order of attribute resolution is order of declaration of parent class
class Myclass(First_parent_class, Second_parent_class, ...)
bases
Return direct parent (super) classes of object (not instance)
class A: pass
class B: pass
class C(A, C): pass
print(D.__bases__)
>>> (<class '__main__.A'>, <class '__main__.C'>)
example
a = 1
class C():
a = 2
class B():
def f(self):
print(a)
print(C.a)
c = C.B()
c.f()
class A():
def __init__(self):
print('A')
class B(A):
def __init__(self):
print('B')
class C(B):
def __init__(self):
super().__init__()
print('C')
class D(C):
def __init__(self):
super(B, self).__init__()
print('D')
a = A()
>>> A
b = B() # not call parent
>>> B
c = C() # from parent
>>> B
>>> C
d = D() # from higher ancestor than parent
>>> A
>>> D
Method Resolution Order (MRO)
Left-Top
It traverses the inheritance tree from bottom to top and then from left to right, keeping only the last instance of each class appearing in the inheritance tree.
example
class A1: pass
class B1: pass
class C1: pass
class A2(A1, C1): pass
class B2(B1): pass
class C3(A2, B2, C1): pass
c3 = C3()
print(C3.mro())
>>> [<class '__main__.C3'>, <class '__main__.A2'>, <class '__main__.A1'>, <class '__main__.B2'>, <class '__main__.C1'>, <class '__main__.B1'>, <class 'object'>]
Composition
class Engine:
def start(self):
print("Engine started")
class Car:
def __init__(self):
self.engine = Engine() # Composition: Car contains an Engine
def start(self):
self.engine.start()
print("Car moving")
my_car = Car()
my_car.start()
example
class Sentence:
_call_counter = 0
@staticmethod
def call_counter():
return Sentence._call_counter
def __init__(self, sentence = ''):
Sentence._call_counter += 1
self.sentence = sentence
def __repr__(self):
return f"{__class__} called {Sentence.call_counter()}"
class SentenceLower(Sentence):
def __init__(self, sentence = ''):
SentenceLower._call_counter += 1
self.sentence = sentence.lower()
s1 = Sentence("ma phrase s1") # add 1 to Sentence._call_counter
s2 = SentenceLower("ma phrase s2") # add 1 to SentenceLower._call_counter
s3 = SentenceLower("ma phrase s3") # add 1 to SentenceLower._call_counter
print(Sentence._call_counter)
>>> 2 # s1 & s2 add one each
print(SentenceLower._call_counter)
>>> 3 # s2 & s3 add one each but s2 inherit of s1: (1 + 1) + 1
print(s3._call_counter)
>>> 3 # s3 inherit of SentenceLower._call_counter
dataclass
https://docs.python.org/3/library/dataclasses.html
Record data class
frozen=True -> unmutable
examples
from dataclasses import dataclass
>>>
@dataclass(frozen=True)
class Point:
x: float
y: float
def __eq__(self, other):
return self.x == other.x and self.y == other.y
def __hash__(self):
return (11 * self.x + self.y) // 16
p1, p2, p3 = Point(1, 1), Point(1, 1), Point(1, 1)
s = {p1, p2}
len(s)
p3 in s
try:
p1.x = 10
except Exception as e:
print(f"OOPS {type(e)}")
from dataclasses import dataclass
@dataclass(frozen = True)
class Personne:
nom: str
age: int
ville: str = "Paris" # Valeur par défaut
# Utilisation
p = Personne("Alice", 30)
p2 = Personne("Joe", 35)
print(p) # Affiche : Personne(nom='Alice', age=30, ville='Paris')
s = {p, p2}
print(s)
print(type(p))
print(vars(p))
#print(vars(Personne))
namedtuple
The objects will be unmutable (in fact they are tuples)
Incidentally we can access the different fields by their name as well as by an index
example
from collections import namedtuple
TuplePoint = namedtuple('TuplePoint', ['x', 'y'])
p3 = TuplePoint(1, 2)
p3.x
p3[0]
class Point2(namedtuple('Point2', ['x', 'y'])):
# l'égalité va se baser naturellement sur x et y
def __eq__(self, other):
return self.x == other.x and self.y == other.y
# du coup la fonction de hachage
# dépend aussi de x et de y
def __hash__(self):
return (11 * self.x + self.y) // 16
q1 is q2
q1 == q2
s = {q1, q2}
q3 in s
# unmutable
try:
q1.x = 100
except Exception as e:
print(f"OOPS {type(e)}")https://docs.python.org/3/reference/lexical_analysis.html#reserved-classes-of-identifiers
https://docs.python.org/3/tutorial/classes.html#tut-private
- A Class & class instance is mutable
- Inheritance tree: instance -> class -> parent class....
- Attributes are calculate dynamicaly in time
- Instance & class are shared same attributes
- Self for instance is just a convention
class Test:
att = 0
t = Test
s = "Here is !"
class Myclass:
def initia(self, s):
self.word = s
p = Myclass()
p.initia(s)
Myclass.initia(p,s)
vars & dir
# return space name
vars(Myclass) <=> Myclass.__dict__
# return attributs & methods
dir(Myclass) <=> Myclass.__dir__()
Properties
https://docs.python.org/3.6/library/functions.html#property
To use an encapsulation with attributes, use property to define the setters or getters, the calling attributes launch automatically the setters or getters
Attributes Class vs instance
- Class attribute defined in class are shared with all instances
- Attributes reference are shared & changed according to whether it is mutable or not
class Point:
a = 0
b = 0
d = [a, b]
def __init__(self, x=0, y=0, len=0):
self.x = x
self.y = y
self.p = [x, y]
def __repr__(self):
return f"p[{self.x}, {self.y}]"
# class attribute
t = Point()
print(f"t.d={t.d}", f"t.p={t.p}", f"Point.d={Point.d}", sep=" - ")
>>> t.d=[0, 0] - t.p=[0, 0] - Point.d=[0, 0]
try: Point.p
except AttributeError as e: print(f"{type(e)}: {e}")
>>> <class 'AttributeError'>: type object 'Point' has no attribute 'p'
# class -> instance
Point.a = 1
t2 = Point()
print(f"Point.a={Point.a}", f"t.a={t.a}", f"t2.a={t2.a}", sep=" - ")
>>> Point.a=1 - t.a=1 - t2.a=1
# attribute not mutable: instance !-> class
# loose reference
t2.a = 2
print(f"Point.a={Point.a}", f"t.a={t.a}", f"t2.a={t2.a}", sep=" - ")
>>> Point.a=1 - t.a=1 - t2.a=2
Point.a = 3
print(f"Point.a={Point.a}", f"t.a={t.a}", f"t2.a={t2.a}", sep=" - ")
>>> Point.a=3 - t.a=3 - t2.a=2
# attribute mutable: instance -> class
t2.d.append(3)
print(f"Point.d={Point.d}", f"t.d={t.d}", f"t2.d={t2.d}", sep=" - ")
>>> Point.d=[0, 0, 3] - t.d=[0, 0, 3] - t2.d=[0, 0, 3]
Point.d.append(4)
print(f"Point.d={Point.d}", f"t.d={t.d}", f"t2.d={t2.d}", sep=" - ")
>>> Point.d=[0, 0, 3, 4] - t.d=[0, 0, 3, 4] - t2.d=[0, 0, 3, 4]
## instance attribute
t = Point()
print(f"t.p={t.p}", f"t2.p={t2.p}", sep=" - ")
>>> t.p=[0, 0] - t2.p=[0, 0]
t.p.append(1)
print(f"t.p={t.p}", f"t2.p={t2.p}", sep=" - ")```
>>> t.p=[0, 0, 1] - t2.p=[0, 0]
### instance equality
**instances are compared by the id not values**
```python
p1 = Point1(2, 3)
p2 = Point1(2, 3)
p3 = Point1(2, 3)
Equality test instance id, not class values
p1 == p2
In test instance id, not class values
s = {p1, p2}
p1 in s
p3 in s
See examples in class-special-methods for hash eq__
getattr(), setattr(), hasattr()
# Retrieves the value of an attribute of an object
getattr(object, name[, default])
# Sets or modifies the value of an object's attribute
setattr(object, name, value)
# Checks if an object has a given attribute
hasattr(object, name)
import math
math.pi
hasattr(math, 'pi')
getattr(math, 'pi')
getattr(math, 'pi2', 'not found')
setattr(math, 'pi', math.pi*2)
math.pi
dynamic overload
class Test:
class_attr = 0
def __init__(self, value = 0):
self.attr = value
def class_add(value = 0):
Test.class_attr += value
def instance_add(self, value):
self.attr += value
dir(Test)
Test.class_add = class_add
Test.instance_add = instance_add
dir(Test)
Test.class_add(10)
print(Test.class_attr)
>>> 10
t = Test()
t.instance_add(100)
print(t.attr)
>>> 100
Enum
https://docs.python.org/3/library/enum.html
enumeration
from enum import Enum
class Flavour(Enum):
CHOCOLATE = 1
VANILLA = 2
PEAR = 3
chocolate = Flavour['CHOCOLATE']
chocolate
chocolate.name
IntEnum
https://docs.python.org/3/library/enum.html
IntEnum is Enum with comparison
from enum import IntEnum
class HttpError(IntEnum):
OK = 200
REDIRECT = 301
REDIRECT_TMP = 302
NOT_FOUND = 404
INTERNAL_ERROR = 500
def is_redirect(self):
return 300
examples
class Redirector:
def __init__(self, id):
self.id = id
def __repr__(self):
return f"Redirector({self.id})"
def __getattr__(self, name):
def forwarder(arg):
return f"{self.id} -> {name}({arg})"
return forwarder
class Quaternion:
'''Dictionary for representation'''
QR = {0:'', 1:'i', 2:'j', 3:'k'}
'''
Array for quaternion multiplication with this references for the result: 1:a 2:i 3:j 4:k
start at 1 to allow negative values (for integer part)
'''
MUL = ( # * 1 i j k
(1, 2, 3, 4), # 1 1 i j k
(2, -1, 4, -3), # i i -1 k -j
(3, -4, -1, 2), # j j -k -1 i
(4, 3, -2, -1) # k k j -i -1
)
@property
def quaternion(self):
return self._quaternion
@quaternion.setter
def quaternion(self, quaternion):
self._quaternion = quaternion
def __init__(self, *args):
if len(args) == 4:
self.quaternion = args
elif len(args) == 1:
args = args[0]
if isinstance(args, int):
self.quaternion = (args, 0, 0, 0)
if isinstance(args, complex):
self.quaternion = (int(args.real), int(args.imag), 0, 0)
else:
raise TypeError(f"Attribut must be a class: {tuple} of 4 int, {int} or {complex} not: {type(args)}")
def __add__(self, arg):
if isinstance(arg, complex) or isinstance(arg, int):
arg = Quaternion(arg)
if isinstance(arg, Quaternion):
new = [i + j for i, j in zip(self._quaternion, arg._quaternion)]
return Quaternion(*new)
else:
raise TypeError(f"Attribut must be a class: {Quaternion}, {int} or {complex} not: {type(arg)}")
def __radd__(self, arg):
return self.__add__(arg)
def __eq__(self, arg):
if isinstance(arg, complex) or isinstance(arg, int):
arg = Quaternion(arg)
if isinstance(arg, Quaternion):
return all(i == j for i, j in zip(self._quaternion, arg._quaternion))
else:
raise TypeError(f"Attribut must be a class: {Quaternion}, {int} or {complex} not: {type(arg)}")
def __mul__(self, arg):
if isinstance(arg, complex) or isinstance(arg, int):
arg = Quaternion(arg)
if isinstance(arg, Quaternion):
if self._quaternion == (0, 0, 0, 0) or arg._quaternion == (0, 0, 0, 0):
return Quaternion(0, 0, 0, 0)
result = [0] * 4
for es, cs in enumerate(self._quaternion):
for ea, ca in enumerate(arg._quaternion):
index, coeff = self.get_mul(es, ea, cs, ca)
result[index] += coeff
return Quaternion(*result)
else:
raise TypeError(f"Attribut must be a class: {Quaternion}, {int} or {complex} not: {type(arg)}")
def get_mul(self, es, ea, cs, ca):
if cs == 0 or ca == 0:
return (0, 0)
else:
index = Quaternion.MUL[es][ea]
coef = cs * ca if index > 0 else -(cs * ca)
# remove one because Quaternion.MUL give as result + 1
return (abs(index) - 1, coef)
def __repr__(self):
result = [('+' if c > 0 else '') + str(c) + Quaternion.QR[e] for e, c in enumerate(self.quaternion) if c]
result = ''.join(result).lstrip('+')
return f"{result:.1f}" if isinstance(result, float) else f"{result}"```https://docs.python.org/3/reference/simple_stmts.html#the-import-statement
module are:
- are mutable !!
- define a namespace
- is a .py file
- import are only made on time during a program
- the file are searched in order :
1 current path
2 in PYTHONPATH (get it from module os.environ['PYTHONPATH'])
3 in standard library paths: sys.path (from module sys). this list can be modified during execution
IMPORT
Absolute Import
Imports a module by specifying its full path from the root directory
import <module>
- import as
- absolute import
- the file are searched first in sys.path (after in local path)
# alias
import module_name as module_aliased
# example
import math, numpy as np, itertools
from <module> import <attribut>
# example
from itertools import cycle, permutation
from django.conf.urls import *
import_module('module' + 'name')
from importlib import import_module
imported_modtools = import_module('mod' + 'tools')
import modtools
imported_modtools is modtools
Relative Import
Imports a module using periods (.) to indicate its relative position with respect to the current file
. local
.. parent
... grandparent
...
- local path first (before look in sys.path)
- import global module with namespace reference
- abstract the calling !!
- same time of loading than absolute import (all module are loading even)
current directory or package
import a module from the same directory (or package)
from . import object as alias
sub directory or package
import a module from a sub directory (or package)
from .name import object as alias
- local parent path first
- import global module with namespace reference
parent directory or package
works only if the script is executed as a module or called
like python -m subfolder.my_script, not as a direct script: python my_script.py
calls another script which calls the script to function correctly (subfolder.my_script contains relative import)
Not works if the level called is the top level, simply call directly the object
import a module from a parent directory (or package)
from .. import name as module_alias
import just an object from a module (or package) 'name' in a parent directory
# import selected object in module without entire module
from ..name import object_name as object_othername
entry point
for the entry point of program, _name__ return main and not the filename !
So you can find in the main file
if __name__ == "__main__":
....
Let program make specific codes when apply file is calling directly from interpreter & not from an import when calling a file directly from the interpreter, the entry point sets name to 'main', so it s almost impossible to use relative imports in this situation
To use test environment:
python3 -m unittest package_name.module_name
http://sametmax.com/un-gros-guide-bien-gras-sur-les-tests-unitaires-en-python-partie-1/
Show module
from modtools import show_module
import itertools
show_module(itertools.count)
Importlib
Use a variable for module name
from importlib import import_module
modulename = "ma" + "th"
loaded = import_module(modulename)
math is loaded
Import math <=> math = import_module('math')
from pathlib import Path
<=>
tmp = import_module('pathlib')
Path = tmp.Path
del tmp
Reload
Force to reload manualy a module
from importlib import reload
reload(MODULE)
Attibutes
https://docs.python.org/3/tutorial/modules.html#importing-from-a-package
# Module name
__name__
# The complete path of package definition file: __init__.py
__file__
# redefine the behavior of import *
__all__
Path
https://docs.python.org/3/tutorial/modules.html#the-module-search-path
Environnement variable
import os
os.getcwd() # return current working directory
os.listdir(path) # return the list of path & files in path (if empty return current path)
os.environ # dict of full os informations
os.environ['PATH'] os.environ['PYTHONPATH']
sorted([i for i in os.environ])
Include paths
import sys
sys.path
Add path when launch python script
PYTHONPATH=/path python script.py
sys.modules
https://docs.python.org/3/library/sys.html#sys.modules
examples
import sys
# test on loaded modules
'csv' in sys.modules
import csv
'csv' in sys.modules
csv is sys.modules['csv']
# unload module
del sys.modules['multiple_import']
# A tuple of strings giving the names of all modules that are compiled into this Python interpreter
sys.builtin_module_names
# modules.keys() only lists the imported modules
sys.modules.keys()
'csv' in sys.modules.keys()
cwd
from pathlib import Path
Path.cwd()
Import file from anywhere
from pathlib import Path
directory_installation = Path(__file__).parent
import sys
sys.path.append(directory_installation)
modtools
Show_module
from modtools import show_module
import math
show_module(math)
setuptools / pypi
https://pypi.org/project/setuptools/
https://pypi.org/
Deploy setuptools
pip3 install setuptools
package
https://docs.python.org/3/tutorial/modules.html#packages
A package are:
- a folder with the same name
- a namespace
- this folder have to contains a mandatory file init.py
import package
import package_name
import package_name as package_alias
Use namespace
package_name.object
not recommended
package_name.module_name.object
subpackage
Package physically inside another
import package_name.subpackage_name
import package_name.subpackage_name.module_name.element_name
examples
import this
def decode_zen(this):
return ''.join(this.d[i] if i in this.d else i for i in this.s)
# better
def decode_zen(this):
return ''.join(this.d.get(i) for i in this.s)LAMBDA
Anonym function, to use as an expression
from functools import reduce
from operator import add
def factoriel(x):
return reduce(lambda x, y: x*y, range(1, x+1), 1)
def fibonacci(x):
return reduce(operator.add, range(1, x+1), 0)
def fibonacci(x):
return reduce(add, range(1, x+1), 0)
With comprehension
[(lambda x: x**2)(x) for x in range(10) if x % 2 == 0]
MAP
Return an iterator that applies function to every item of iterable, yielding the results
list(map(lambda x: x**2, filter(lambda x: x % 2 == 0, range(10))))
FILTER
Construct an iterator from those elements of iterable for which function returns true
list(filter(lambda x: x % 2 == 0, range(10)))
EXERCISE
1-1
More efficient, don't duplicate listes
def multi_tri(listes):
for liste in listes:
liste.sort()
return listes
def multi_tri(listes):
return list(map(sorted, listes))
1-2
More efficient, don't duplicate listes
def multi_tri_reverse(listes, reverses):
for x,y in zip(listes, reverses):
x.sort(reverse=y)
return listes
def multi_tri_reverse(listes, reverses):
return [sorted(x, reverse=y) for x,y in zip(listes, reverses)]
def multi_tri_reverse(listes, reverses):
return list(map(lambda x, y: sorted(x, reverse=y), listes, reverses))
2-1
def doubler_premier(f, arg, *args):
return f(2*arg,*args)
def doubler_premier(f, arg1, *args):
from functools import reduce
from operator import add, mul
return reduce(
f,
args + (arg1*2,),
1 if f is mul else 0
)
2-2
def doubler_premier_kwds(f, arg, *args, **kwargs):
return f(2*arg, *args, **kwargs)
def doubler_premier_kwds(f, arg1, *args, y=0, z=0):
from functools import reduce
return reduce(
f,
[arg for arg in args + (arg1*2,) + (y,) + (z,) if arg],
1 if f.__name__ == 'mul3' else 0
)
3-1
def compare_all(f, g, entrees):
return [f(entree) == g(entree) for entree in entrees]
3-2
def compare_args(f, g, entrees):
return [f(*entree) == g(*entree) for entree in entrees]https://docs.python.org/fr/3.7/howto/functional.html
https://docs.python.org/3/library/itertools.html
ITERATOR
- iterator allow to calculate data of function only on read it
- builtin function all methods on object
s = {0, 1, 'string1', 3, 'string2',5 , 6}
[x for x in s if type(x) is int]
i = iter(s)
type(i)
dir(i)
next(i)
i.__next__()
ITERABLE
- Object contains a method 'iter', which gives iterator methods
- An object wich on you can use iterator methods
ITERTOOLS
https://docs.python.org/3/library/itertools.html
This module implements a number of iterator building blocks
import itertools
Infinite iterators
count(start, [step])
start, start+step, start+2*step, …
count(10) --> 10 11 12 13 14 ...
cycle(p)
p0, p1, … plast, p0, p1, …
cycle('ABCD') --> A B C D A B C D ...
repeat(elem [,n])
elem, … endlessly or up to n times
repeat(10, 3) --> 10 10 10
Iterators terminating on the shortest input sequence
accumulate(p[,func])
p0, p0+p1, p0+p1+p2, …
accumulate([1,2,3,4,5]) --> 1 3 6 10 15
chain(p, q, …)
p0, p1, … plast, q0, q1, …
chain('ABC', 'DEF') --> A B C D E F
chain.from_iterable(iterable)
p0, p1, … plast, q0, q1, …
chain.from_iterable(['ABC', 'DEF']) --> A B C D E F
compress(data, selectors)
(d[0] if s[0]), (d[1] if s[1]), …
compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F
dropwhile(pred, seq)
seq[n], seq[n+1], starting when pred fails
dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1
filterfalse(pred, seq)
elements of seq where pred(elem) is false
filterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8
groupby(iterable[, keyfunc])
sub-iterators grouped by value of keyfunc(v)
islice(seq, [start,] stop [, step])
elements from seq[start:stop:step]
islice('ABCDEFG', 2, None) --> C D E F G
starmap(func, seq)
func(seq[0]), func(seq[1]), …
starmap(pow, [(2,5), (3,2), (10,3)]) --> 32 9 1000
takewhile(pred, seq)
seq[0], seq[1], until pred fails
takewhile(lambda x: x<5, [1,4,6,4,1]) --> 1 4
tee(it, n)
it1, it2, … itn splits one iterator into n
zip_longest(p, q, …)
(p[0], q[0]), (p[1], q[1]), …
zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
Combinatoric iterators
product(p, q, … [repeat=1])
cartesian product, equivalent to a nested for-loop
product('ABCD', repeat=2) -> AA AB AC AD BA BB BC BD CA CB CC CD DA DB DC DD
permutations(p[, r])
r-length tuples, all possible orderings, no repeated elements
permutations('ABCD', 2) -> AB AC AD BA BC BD CA CB CD DA DB DC
combinations(p, r)
r-length tuples, in sorted order, no repeated elements
combinations('ABCD', 2) -> AB AC AD BC BD CD
combinations_with_replacement(p, r)
r-length tuples, in sorted order, with repeated elements
combinations_with_replacement('ABCD', 2) -> AA AB AC AD BB BC BD CC CD DD
MORE ITERTOOLS
EXPRESSION
https://docs.python.org/fr/3.7/howto/functional.html
create an generator of iterator in place of comprehension
(treatment for ... if ...)
carre = (x**2 for x in range(1000))
palin = (x for x in carre if str(x) == str(x)[::-1])
carre
palin
# only now, calculate
list(palin)
GENERATOR FUNCTION
http://python-history.blogspot.fr/2010/06/from-list-comprehensions-to-generator.html
def carre(a, b):
for i in range(a, b):
yield i**2
def palin(it):
for i in it:
if (isinstance(i, (str, int)) and
str(i) == str(i)[::-1]):
yield i
list(palin(x**2 for x in range(1000)))
EXCERCISE
def produit_scalaire(X, Y):
return sum(x*y for x, y in zip(X, Y))
g = produit_scalaire((1, 2), (3, 4))https://docs.python.org/3/tutorial/datastructures.html#list-comprehensions
types
# list
l = [i for i in iter]
# generator
g = (i for i in iter)
# set
s = {i for i in iter}
# dict
d = {k:v for k, v d.items()}
use variables
accumulate_items = 0
[accumulate_items := accumulate_items + i for i in range(10)]
accumulate_items = 0
sum(accumulate_items := accumulate_items + i for i in range(10))
nesting
flat nesting : 1 level => [n(p(..., ...) ,...)]
[... for ... for ...]
[... > ... > ...]
for n ...
for p ...
...
[str(n) + str(p) for n in [2, 4] for p in [10, 20, 30]]
classic "sub"
[... for n in [... p ...] if ...]
for p ...
for n ...
embedded "sub" nesting 2 levels => [p[n... ], ...]
[[... for ...] for ...]
[[... < ...] < ...]
for p ...
for n ...
[[str(n) + str(p) for n in [2, 4]] for p in [10, 20, 30]]
flat
list = [[sublist], [sublist], [sublist], ...]
[item for sublist in list for item in sublist]
for sublist in list
for item in sublist
item
repeat
# compare:
[x + y for x in (1,2,3) for y in (10,20) for z in (0,2)]
[x + y for x in (1,2,3) for z in (0,2) for y in (10,20)]
[x + y for x in (1,2,3) for z in (0,2) for y in (10,20)]
t = (y for y in (x**2 for x in range(1000)) if str(y) == str(y)[::-1])
restriction
[str(n) + str(p) for n in [2, 4] for p in [10, 20, 30] if n*p >= 40]
break
def end_of_loop():
raise StopIteration
even = list(end_of_loop() if n == 412 else n for n in numbers if 0 == n % 2)
even = list(next(iter(())) if n == 412 else n for n in numbers if 0 == n % 2)
examples
# flat vs nested
[[str(n) + str(p) for n in (2, 4)] for p in (10, 30, 50)]
[str(n) + str(p) for p in (10, 30, 50) for n in (2, 4)]
# flat vs nested
[[str(n) + str(p) for n in (2, 4)] for p in (10, 30, 50)]
[str(n) + str(p) for p in (10, 30, 50) for n in (2, 4)]
def aplatir(conteneurs):
return [c2 for c1 in conteneurs for c2 in c1]
def aplatir(conteneurs):
r=[]
for c1 in conteneurs:
for c2 in c1:
r.append(c2)
return r
aplatir(((1, [2, 3]), [], 'a', ['b', 'c']))
def alternat(c1, c2):
return [c_2 for c_1 in zip(c1, c2) for c_2 in c_1]
alternat( (1, 2, 3), ('a', 'b', 'c'))
def intersect(A, B):
return {c2 for c1 in ((v1, v2) for i1, v1 in A for i2, v2 in B if i1 == i2) for c2 in c1}
intersect({ (1, 'unA'), (2, 'deux'), (3, 'troisA')}, { (1, 'unB'), (2, 'deux'), (4, 'quatreB')})
# more elegante
def intersect(A, B):
return {v for dictionnaire in [dict(A),dict(B)] for k,v in dictionnaire.items() if k in dict(A) and k in dict(B)}
def intersect(A, B):
return { i for item in (dict(A).keys() & dict(B).keys()) for i in (dict(A)[item], dict(B)[item]) }SCOPE
https://www.programiz.com/python-programming/namespace
Order to interprate variables is LEGB
- Local
- Enclosing
- Global
- Builtin
locals()
return a list of actual local scope variables
globals()
return a list of actual global scope variables
Global scope & function error !!
Because var is referenced like global by "print(var)" & use like local with "var = 'local'"
Solution: remove "print(var)" or remove "var = 'local'" or put it after "var = 'local'"
var = 'global'
def f():
print(var)
var = 'local'
f()
print(var)
GLOBAL
to use global variable locally
var = 'global'
def f():
global var
var = 'local'
print(var)
f()
print(var)
NONLOCAL
- To use upper local scope variable locally
- The upper scope local variable is the closest one
- A nonlocal variable have to be a binding variable in upper local scope (in function definition, not in the global scope) else an exception are called
a = 'a global'
def f():
a = 'a f'
def g():
nonlocal a
a = 'a g'
print(a)
g()
print(a)
f()
print(a)
BUILTINS
import builtins
modules with all builtin functions
dir(builtins)
dir(builtins.str)
To ensure to call a really builtin function
import builtins
with builtins.open(...) as f:
GLOBAL VARIABLES
vars()
globals()
To access object to other namespace
print(dir(spam))
print(dir(spam.__builtins__))GENERAL
https://docs.python.org/3/library/stdtypes.html#truth-value-testing
https://docs.python.org/3/reference/expressions.html#conditional-expressions
http://legacy.python.org/dev/peps/pep-0308/
https://docs.python.org/3/tutorial/datastructures.html#more-on-conditions
Order to return test:
- call bool
- call len
bool(object)
- object.bool()
- object.len()
type built-in
false : False None [] {} () ''
true : others
INSTRUCTION / EXPRESSION
For if you can use any of expressions
Instructions Expressions
assignment function call
import operators is, in, ==, ...
instruction if conditional expression
instruction for List Comprehensions
syntax
if condition:
...
elif condition:
...
else:
...
inputs = [23, 65, 24]
def condition(n):
return (n**2) % 10 == 5
if [value for value in inputs if condition(value)]:
print("au moins une entrée convient")
FEW OPERATORS TEST IN LINE
if 0 < a < 10 < b:
instruction
TERNARY OPERATORS & SHORT TEST
(result_if_false, result_if_true)[condition]
Result_if_true if condition else result_if_false # ternary operators
is_nice = True
if is_nice:
state = "nice"
else:
state = "not nice"
<=>
# ternary operators
state = "nice" if is_nice else "not nice"
<=>
# short test
state = ("not nice", "nice")[is_nice]
print(state)
SHORT-CIRCUIT
if the first tests are sufficient the following ones are not evaluated
# stop at first test
false and true
true or false
NESTING
valeur = -1 if x < -10 else (0 if x <= 10 else 1)
BOOL
Use bool function to test the behavior of return
def show_bool(x):
print(f"condition {repr(x):>10} considérée comme {bool(x)}")
for exp in [None, "", 'a', [], [1], (), (1, 2), {}, {'a': 1}, set(), {1}]:
show_bool(exp)
OPERATORS
| Famille | |Exemples | |---------|-|---------| | Égalité | |==, !=, is, is not | | Appartenance | | in | | Comparaison | | <=, <, >, >= | | Logiques | | and, or, not |
PRIORITY ORDER
a and not b or c and d
<=>
(a and (not b)) or (c and d)
Operators can be used with others than boolean
The last non evaluated elements is always return else when the short-circuit can be used
1 and [1, 2]
1 or [1, 2]
1 and 2 and 3
1 and 2 and 3 and '' and 4
[] or "" or {}
[] or "" or {} or 4 or set()
Examples
1
def dispatch1(a, b):
def isimpair(x): return(x%2)
def ispair(x): return(not x%2)
if ispair(a) and ispair(b): return(a**2 + b**2)
elif ispair(a) and isimpair(b): return(a * (b - 1))
elif isimpair(a) and ispair(b): return((a - 1) * b)
else: return(a**2 - b**2)
2
def dispatch2(a, b, A, B):
if a in A and not (b in B): return(a * (b - 1))
elif not (a in A) and b in B: return((a - 1) * b)
else: return(a**2 + b**2)
3
def libelle(ligne):
for i in [" ", "\t"]:
ligne = ligne.replace(i, '')
ligne = ligne.split(',')
if len(ligne) != 3:
return
prenom, nom, rang = ligne
if not rang: rang = "-ème"
elif int(rang) == 1: rang = "1er"
else: rang = f"{rang}-ème"
return(f"{nom}.{prenom} ({rang})")WHILE
Execute the loop while condition is true and execute else:
while condition:
...
else:
...
continue
Step in a loop to the next loop
break
Break the loop definitively
while true
while true, continual loop with a break to exit
while true:
...
break
example
while liste:
element = liste.pop()
print(element)
FOR
for item in element
...
else
...
EXCERCISE
1
def pgcd(a, b):
if a == 0:
return(b)
elif b == 0:
return(a)
while True:
if b > a:
b = b % a
if b == 0:
return(a)
else:
a = a % b
if a == 0:
return(b)
2
def taxes(income):
taxe = 0
if income > 150_000:
taxe += (income - 150_000) * (45/100)
income = 150_000
if income > 45_000:
taxe += (income - 45_000) * (40/100)
income = 45_000
if income > 11_500:
taxe += (income - 11_500) * (20/100)
income = 11_500
return (int(taxe))
2 bis
def taxes(income):
taxe = 0
limit_rates = (
(150_000, 45),
(45_000, 40),
(11_500, 20)
)
for limit, rate in limit_rates:
if income > limit:
taxe += (income - limit) * (rate/100)
income = limit
return (int(taxe))A function performs a variable passing by reference !!
class Test:
"""class Test to make a test with a long text to see differtence"""
def __init__(self):
"""init property listin with an empty list"""
self.listin = []
def add(self, value):
"""add value to listin property"""
self.listin.append(value)
def print(self):
"print listin property to stdout"
print(self.listin)
a = [0]
p.add(a)
p.print()
a[0] = 100
p.print()
doc
https://legacy.python.org/dev/peps/pep-0257/
# Complet help for Test class
help(Class)
# Programming help
Class.__doc__
# return all methods & attributs
vars(Class)
Class.__dict__
# return all methods & attributs without description
dir(list)
l = [1,2,3]
dir(l)
l.__dir__()
isinstance(obj, type)
Return boolean & support class inheritance
class Animal:
def __init__(self, name):
self.name = name
class Mammifere(Animal):
def __init__(self, name):
Animal.__init__(self, name)
isinstance(baleine, Mammifere)
isinstance(baleine, Animal)
types
from types import FunctionType
isinstance(factoriel, FunctionType)
isinstance(len, BuiltinFunctionType)
# list all types
import types
dir(types)
type hints
https://docs.python.org/3/library/typing.html
Since python 3.5, allow optional type hinting
var : int = 0
def fact(n : int) -> int:
list
def foo(x: List[int]) -> List[str]:
iterable
def lower_split(sep: str, inputs : Iterable[str]) -> str:
multi-typing
from typing import Dict, Tuple, List
ConnectionOptions = Dict[str, str]
Address = Tuple[str, int]
Server = Tuple[Address, ConnectionOptions]
def broadcast_message(message: str, servers: List[Server]) -> None:
...
https://docs.python.org/3/library/typing.html#user-defined-generic-types
extended type hints
from typing import NewType
UserId = NewType('UserId', int)
user1_id : UserId = 0
example
from typing import TypeVar, Generic
from logging import Logger
T = TypeVar('T')
class LoggedVar(Generic[T]):
def __init__(self, value: T, name: str, logger: Logger) -> None:
self.name = name
self.logger = logger
self.value = value
def set(self, new: T) -> None:
self.log('Set ' + repr(self.value))
self.value = new
def get(self) -> T:
self.log('Get ' + repr(self.value))
return self.value
def log(self, message: str) -> None:
self.logger.info('%s: %s', self.name, message)parameters orders
- ordered
- named
- *tuple
- **dict
ordered
Ordered (mandatory) arguments have to be in first in list of arguments
def f(arg1, arg2, ...):
...
named (default/optionnal) parameters
named parameters have to be placed after positional parameters
def f(arg1, arg2, arg3=opt, arg4=opt, ...):
...
*tuple
'tuple' parameter catch all undeclared parameters (tuple is unique)
it have to be placed after positional & named parameters
def f(*t):
...
def f(arg1, arg2, arg3=opt, *t):
print(f"arg1: {arg1}\narg2: {arg2}\nothers: {t}")
f('nom', 'prenom', list(range(5)), 'et moi', ['etc'])
With optionnal parameters
def f(nom, prenom='henriette', *d):
print(f"nom: {nom}\nprenom: {prenom}\nd: {d}")
f('louis', 'mom', 20, 'pierre', ['tyt'])
**dict
'dict' parameter catch all undeclared named parameters (dict is unique)
it have to be placed after all parameters
def f(**d):
print(f"d: {d}")
f(tata='mom', tonton='pierre', maman='tyt')
With optionnal parameters
def f(nom, prenom='henriette', **d):
print(10*"=" + f"\nnom: {nom}\nprenom: {prenom}\nd: {d}")
f('louis', 'prenom', tata='mom', tonton='pierre', maman='tyt')
f('louis', tata='mom', tonton='pierre', maman='tyt')
unpacking
Used to send arguments to a function from a list / tuple / set
With a good lenght of arguments !!
def f(a, b):
print(10*"-" + f"\na: {a}\nb: {b}")
l = [10, 'toto']
f(*l)
l = (10, 'toto')
f(*l)
l = set([10, 'toto']);
f(*l)
dictionary sending
Use a dictionnary to send arguments to function
Keys names of dictionnary have to be match with arguments names
def f(a, b):
print(10*"-" + f"\na: {a}\nb: {b}")
{'a': 10, 'b': 'toto'}
f(**d)
Use for print (& wrapping)
pp = {'sep':'-', 'end':' >EOL'}
print(10, 20, **pp)
trap
named argument & tuple argument
https://docs.python.org/3/reference/expressions.html#calls
def foo(a, b, c, d):
print(a, b, c, d)
#good
foo(1, c=3, *(2,), **{'d':4})
# wrong
foo (1, b=3, *(2,), **{'d':4})
do not use mutable object in default argument
https://docs.python.org/3/faq/programming.html#why-are-default-values-shared-between-objects
import random
def ajouter_un_aleatoire(resultats=[]):
resultats.append(random.randint(0, 10))
return resultats
ajouter_un_aleatoire()
ajouter_un_aleatoire()
examples
def distance(*arg):
from math import sqrt
result = sqrt(sum([i**2 for i in arg]))
return result if result != 0 else 0
def numbers(*args):
return (sum(args), min(args, default=0), max(args, default=0))https://www.python.org/dev/peps/pep-3132/
assignment
tuple1 = 1, 2
tuple2 = (1, 2)
tuple3 = 1, 2,
tuple4 = (1, 2,)
Assignment of one element
simple1 = 1,
simple2 = (1,)
Pretty print
tuple1 = (
[1, 2, 3],
[4, 5, 6],
[7, 8, 9],
)
addition
tuple3 = tuple1 + tuple2
tuple1 += (3, 4,)
cast
tuple = tuple(list)
SEQUENCE UNPACKING
En réalité, les seules contraintes fixées par Python sont que:
- Le terme à droite du signe = soit un itérable (tuple, liste, string, etc.) ;
- Le terme à gauche soit écrit comme un tuple ou une liste - notons tout de même que l'utilisation d'une liste à gauche est rare et peu pythonique ;
- Les deux termes aient la même longueur - en tout cas avec les concepts que l'on a vus jusqu'ici, mais voir aussi plus bas l'utilisation de *arg avec le extended unpacking.
Switch variables
a, b = b, a
EXTENDED UNPACKING
1
reference = [1, 2, 3, 4, 5]
a, *b, c = reference
print(f"a={a} b={b} c={c}")
a, b, *_ = reference # '_' is for tell to reader that the following values are not used
print(f"a={a} b={b} c={c}")
2
reference = [1, 2, 3]
_, milieu, _ = reference
print('milieu', milieu)
3
ignored, ignored, right = reference
print('right', right)
Deep
structure = ['abc', [(1, 2), ([3], 4)], 5]
(a, (b, ((trois,), c)), d) = structure
print('trois', trois)
(a, (b, ([trois], c)), d) = structure
print('trois', trois)
Ultra deep
Un exemple très alambiqué avec plusieurs variables *extended
tree = [1, 2, [(3, 33, 'three', 'thirty-three')], ( [4, 44, ('forty', 'forty-four')])]
*_, ((_, *x3, _),), (*_, x4) = tree
print(f"x3={x3}, x4={x4}")
# for
l = [(1, 2), (3, 4), (5, 6)]
for a, b in l:
print(f"a={a} b={b}")
for i, j, k in zip(range(3), range(100, 103), range(200, 203)):
print(f"i={i} j={j} k={k}")
enumerate()
Return an enumerate object with iterator in index and value as value
villes = ["Paris", "Nice", "Lyon"]
for i, ville in enumerate(villes):
print(i, ville)
EXERCISE
1
def comptage(in_filename, out_filename):
import re
with open(in_filename, "r", encoding='utf-8') as f_in:
with open(out_filename, "w", encoding='utf-8') as f_out:
i = 0
for line_in in f_in:
i += 1
# wc = len(re.findall('(\w|\.\.\.+)+',line_in))
wc = len(line_in.split())
cc = len(line_in)
f_out.write(f"{i}:{wc}:{cc}:{line_in}")
2
def surgery(liste):
l=len(liste)
if l in [0, 1]: pass
elif l%2 == 0: liste[1], liste[0] = liste[0], liste[1]
elif l%2 == 1: liste[-2], liste[-1] = liste[-1], liste[-2]
return(liste)https://docs.python.org/3/library/stdtypes.html#set-types-set-frozenset
- Usefull to stock only uniq elements
- Set is mutable but element in set ar to be immutable => unable to create a set of set
- Like distionnary, set are not ordered
INIT
s = set()
s = {key1, key2, ...}
s = set([key1, key2, ...])
METHODS
(key) in s
Return true in key is in s
len(s)
Print length of s
s.clear()
Clear elements of s
s.add()
Add a key in set
s.update(s2)
Merge a set into a set
s.discard(key)
Remove elements, no error if not found
s.remove(key)
Remove elements, launch exception if not found
try:
s.remove(key)
except KeyError as e:
print("text", e)
s.pop()
Remove last or indexed element but there is no order in a set !
s = [1, 2, 3, 4, 5]
while s:
s = s.pop()
print("set", s)
print("set is now empty", s)
OPERATIONS
union
s1 | s2
Return the union of sets
intersection
s1 & s2
Return intersection of sets
difference
s2 - s3
Return difference of two sets
symmetrical difference
s1 ^ s2
AΔB=(A−B)∪(B−A)=(A∪B)-(A∩B)
equality
s1 == s2
inclusion
s1 <= s2
s1 < s2
Disjoint sets
s.isdisjoint(s2)
FROZENSET
Immutable set to serve to key of dictionnary or element of set
init
fs = frozenset()
fs = frozenset([key1, key2, ...])
Available methods
copy()
isdisjoint()
symmetric_difference()
difference()
issubset()
union()
intersection()
issuperset()
EXCERCISE
1
def read_set(filename):
s=set()
with open(filename, "r", encoding='utf-8') as f:
for line in f:
#print(line)
s.update({line.strip()})
return(s)
2
def search_in_set(filename_reference, filename):
s = set()
l = []
with open(filename_reference, "r", encoding='utf-8') as f_ref:
for line in f_ref:
s.update({line.strip()})
with open(filename, "r", encoding='utf-8') as f_test:
for line in f_test:
line = line.strip()
l.append((line, line in s))
return(l)
3
def diff(extended, abbreviated):
# get ships
ships_ext = set([i[0] for i in extended])
ships_abb = set([i[0] for i in abbreviated])
ships_name = {i[0]: i[4] for i in extended}
# get sets
id_ext_abb = ships_ext - ships_abb
id_abb_ext = ships_abb - ships_ext
id_abb_and_ext = ships_ext & ships_abb
# get ships name
name_ext_abb = [ships_name[i] for i in list(id_ext_abb)]
name_abb_and_ext = [ships_name[i] for i in list(id_abb_and_ext)]
# return
return((set(name_ext_abb), set(name_abb_and_ext), set(id_abb_ext)))by default all is reference !
==
Test equality of value
is
Test equality of reference (use copy.deepcopy() to secure it b = a[:] can create mistake with level of mutable)
Copy / Deep Copy
Mutable elements propagate changes to shared references
a = [1, 2]
b = a
a[0] = 'shared'
print(b)
Use shallow copy for one level
b = a[:]
b = copy.copy(a)
Use deep copy for all levels
import copy
b = copy.deepcopy(a)
Variable identifier
https://docs.python.org/3/reference/datamodel.html#objects-values-and-types
id(v)
Return the identifier of variable
a = 3
b = 3
print(id(a), id(b))
b is a
Circular reference
Delete
https://docs.python.org/3/reference/simple_stmts.html#the-del-statement
del(v)
Delete variable
a = 10
del(a)
try:
print('a=', a)
except NameError as e:
print("a n'est pas définie")
delete a part
l = list(range(10))
del l[2:10:2]
l
delete few elements
d = dict(foo='bar', spam='eggs', a='b')
del(d['a'], d['spam'])
Shared assignment
shared reference
b = a = []
cellule = [0]
liste = [cellule, cellule, cellule]
liste[0][0] = 1
print(liste, cellule)
shared assignment
liste = 3 * [[0]]
l[0][0] = 1
print(l)
copy assignment
liste = [[0], [0], [0]]
l[0][0] = 1
print(l)
assignment
+= *= ... # make default assignment by reference <=> make assignment by reference if variable is mutable
mutable
b = a = [1]
a += [1]
print(a); print(b); print(b is a); print(b == a)
not mutable
b = a = 1
a += 1
print(a); print(b); print(b is a); print(b == a)Pathlib
https://docs.python.org/3/library/pathlib.html
exists()
test if file exists
from pathlib import Path
filename = '/tmp/test'
path = Path(filename)
path.exists()
with open(filename, 'w', encoding='utf-8') as f:
f.write('0123456789\n')
path.exists()
stat()
return statistiques about file
path.stat()
st_mode
st_ino
st_dev
st_nlink
st_uid
st_gid
st_size
st_atime
st_mtime
st_ctime
path.stat().st_size
from datetime import datetime
datetime.fromtimestamp(path.stat().st_mtime)
f"{mtime_datetime:%H:%M}"
unlink()
delete file
try:
path.unlink()
except FileNotFoundError:
print("no need to remove")
path = Path('/tmp')
for json in dirpath.glob("*.json"):
print(json)
Class
path = Path('/tmp')
type(path)
from pathlib import PosixPath
issubclass(PosixPath, Path)
isinstance(path, Path)
FILE FORMAT
https://docs.python.org/3/library/fileformats.html
json
https://docs.python.org/3/library/json.html
limitations
Tuple, qui se fait encoder comme une liste / complex, set et frozenset, que l'on ne peut pas encoder du tout (sans étendre la bibliothèque)
pickle
https://docs.python.org/3/library/pickle.html
csv
https://docs.python.org/3/library/csv.html
Sys
import sys
autre_stdout = open('ma_sortie.txt', 'w', encoding='utf-8')
tmp = sys.stdout
print('sur le terminal')
sys.stdout = autre_stdout
print('dans le fichier')
sys.stdout = tmp
autre_stdout.close()
print('de nouveau sur le terminal')
with open("ma_sortie.txt", encoding='utf-8') as check:
print(check.read())
Os
Old fashion
os.path.join()
ajoute '/' ou '' entre deux morceaux de chemin, selon l'OS
os.path.basename()
trouve le nom de fichier dans un chemin
os.path.dirname()
trouve le nom du directory dans un chemin
os.path.abspath()
calcule un chemin absolu, c'est-à-dire à partir de la racine du filesystem
os.path.exists()
pour savoir si un chemin existe ou pas (fichier ou répertoire)
os.path.isfile .isdir()
Pour savoir si un chemin est un fichier (et un répertoire)
os.path.getsize()
pour obtenir la taille du fichier
os.path.getatime()
et aussi getmtime et getctime pour obtenir les dates de création/modification d'un fichier
os.chdir()
change current directory
os.remove()
(ancien nom os.unlink), qui permet de supprimer un fichier
os.rmdir()
pour supprimer un répertoire (mais qui doit être vide)
os.removedirs()
pour supprimer tout un répertoire avec son contenu, récursivement si nécessaire
os.rename()
pour # renommer un fichier
glob.glob()
comme dans par exemple glob.glob("*.txt")
https://docs.python.org/3/library/functions.html#open
options
r # open for reading (default)
w # open for writing, truncating the file first
x # open for exclusive creation, failing if the file already exists
a # open for writing, appending to the end of the file if it exists
b # binary mode
t # text mode (default)
+ # open a disk file for updating (reading and writing)
U # universal newlines mode (deprecated)
# open(file, mode, buffering, encoding, errors, newline, closefd, opener)
f = open('file', 'options'[, encoding=, ...])
f.close()
encoding
Necessary if you don't use option binary 'b'
f = open('/tmp/spam', 'w', encoding='utf8')
f.close()
f = open('/tmp/spam', 'bw')
f.close()
CONTEXT MANAGER
Close automaticaly file after treatment and exception
# writing
with open("foo.txt", "w", encoding='utf-8') as f:
for i in range(2):
f.write(f"{i}\n")
# reading
with open("foo.txt", "r", encoding='utf-8') as f:
for line in f:
print(line, end='')
iterator
with open("foo.txt", encoding='utf-8') as f:
print(f.__iter__() is f)
ADVANCED
repr()
View real content
with open("foo.txt", encoding='utf-8') as f:
for bloc in range(2):
print(f"Bloc {bloc} >>{repr(f.read(4))}<<")
read()
Read all content of file
with open("foo.txt", encoding='utf-8') as entree:
full_contents = entree.read()
print(f"Contenu complet\n{full_contents}", end="")
flush()
Force writing to disk
binary mode
with open('strbytes', 'w', encoding='utf-8') as output:
output.write("déjà l'été\n")
with open('strbytes', 'rb') as rawinput:
octets = rawinput.read()
print("on a lu un objet de type", type(octets))
for i, octet in enumerate(octets):
print(f"{i} → {repr(chr(octet))} [{hex(octet)}]")
Difference between characters & octets
with open('strbytes', encoding='utf-8') as textfile:
print(f"en mode texte, {len(textfile.read())} caractères")
with open('strbytes', 'rb') as binfile:
print(f"en mode binaire, {len(binfile.read())} octets")dict
https://docs.python.org/3/library/stdtypes.html#mapping-types-dict
- Dict allow to associate keys and values
- Keeps only uniq keys
- key of dict have to be globally immutable
init
Syntax
d = {}
d = dict()
d = {key1: value1, key2: value2, ...}
d = dict([(key1, value1), (key2, value2), ...])
d = dict(key1=value1, key2=value2, ...])
assign
dic[key]
dic.get(key, index)
merge
x = {'a':1, 'b':2}
y = {'b':3, 'c':4}
z = {**x, **y}
print(z)
>>> {'a': 1, 'b': 3, 'c': 4}
del(d), del(d[k])
del(d[key]) # delete a key
del(d) # delete the entire dictionary
existence test
k in d # k in keys
k in d.keys() # k in keys
v in d.values() # v in values
(k, v) in d.items() # pair (key, value) in dictionary
items()
[(k, v) for k, v in d.items()] # return list of tuples (k, v)
#### keys()
```python
[k for k in d.keys()] # return list of keys
values()
[v for v in d.values()] # return list of values
update()
d.update(d2) # update d with values from d2
d.update({'key1':value1, 'key2':value2}) # update d with values
setdefault(key, value)
d.setdefault(key, value) # only append a tuple if not already exists
Collections
namedtuple
Named tuples are very similar to tuples, from which they inherit. They simply add the ability to reference tuple fields by name rather than by index, for improved readability.
Point2D = namedtuple('Point2D', ['x', 'y'])
p = Point2D(3, 5)
p.x
>>> 3
p.y
>>> 5
p[0]
>>> 3
deque
'collections.deque' (double-ended queue) is a data structure optimized for adding or removing elements from both ends (left or right). It is more efficient than lists for these operations (O(1) time).
usefulness
- Queues (FIFO) or stacks (LIFO)
- Algorithms such as BFS (Breadth-first search)
- Management of buffers or circular logs
Limitations
- Random access is slower than a list (d[0] is O(1), but d[i] is O(n) for i far from the edges)
- Less suitable for searching or sorting
Main features
- Thread-safe: suitable for queues in multi-threaded environments.
- Dynamic size: can automatically expand or shrink.
Key methods
append(x) # adds x to the right
appendleft(x) # adds x to the left
pop() # removes and returns the element on the right
popleft() # removes and returns the element on the left
extend(iterable) # adds multiple elements to the right
extendleft(iterable) # adds multiple elements to the left (in reverse order)
rotate(n) # rotates the elements by n positions (positive = right, negative = left)
clear() # clears the deque
example
d = deque([1, 2, 3])
d.append(4)
d.appendleft(0)
>>> deque([0, 1, 2, 3, 4])
The deques can also be limited in size by removing the rightmost elements when inserting them from the left, and vice versa. This allows for the creation of circular stamps.
d = deque([2], 2)
d.append(3)
>>> deque([2, 3], maxlen=2)
d.appendleft(1)
>>> deque([1, 2], maxlen=2)
d = deque(range(10), 7)
>>> deque([3, 4, 5, 6, 7, 8, 9], maxlen=7)
d.rotate(2)
>>> deque([8, 9, 3, 4, 5, 6, 7], maxlen=7)
d.rotate(-3)
>>> deque([4, 5, 6, 7, 8, 9, 3], maxlen=7)
chainmap
ChainMap allows to group multiple dictionaries or mappable objects into a single logical view.
modifications on chainmap are also made in originals dictionary
usefulness
- Overlapping contexts (for example, default and specific parameters)
- Preventing dictionary duplication while combining their keys
- Prioritizing access: keys are searched in the order of the provided dictionaries
d1 = {'a': 1, 'b': 2}
d2 = {'b': 3, 'c': 4}
c = ChainMap(d1, d2)
c['a'] = 10
print(d1, d2, '/', c)
>>> {'a': 10, 'b': 2} / ChainMap({'a': 10, 'b': 2}, {'b': 3, 'c': 4})
print(c['b'])
>>> 2
print(d1, d2, '/', c)
>>> {'a': 10, 'b': 2, 'c': 6} {'b': 3, 'c': 4} / ChainMap({'a': 10, 'b': 2, 'c': 6}, {'b': 3, 'c': 4})
c.new_child({'v':1, 'w':2})
print(c)
>>> ChainMap({'v': 1, 'w': 2}, {'a': 10, 'b': 2, 'c': 6}, {'b': 3, 'c': 4})
c.parents
>>> ChainMap({'b': 3, 'c': 4})
dict(c)
>>> {'b': 2, 'c': 6, 'a': 10}
counter
'collections.Counter' is a subclass of dict specifically designed to count hashable objects (such as elements in a list, words in a text, etc.). It is optimized for counting operations and offers practical methods.
usefulness
- Count the occurrences of elements in a sequence (list, string, etc.)
- Find the most frequent elements
- Perform mathematical operations between counters (addition, subtraction, intersection)
Key methods
most_common(n) # Returns the n most frequent elements
elements() # Returns an iterator over the repeated elements based on their count
update(iterable) # Updates the counter with the elements of an iterable
subtract(iterable) # Subtracts the counts from an iterable
# Addition: elements present in counter1 and in counter2
counter1 + counter2
# Subtraction: elements present in counter1 but not in counter2
counter1 - counter2
# Intersection: minimum count
counter1 & counter2
# Return only positive counts
+c
# Return only -(negative counts)
-c
Common Error Cases
Non-hashable elements: If you try to count a non-hashable element (such as a list), a TypeError will be thrown
example
c = Counter([1, 2, 3, 1, 3, 1, 5])
>>> Counter({1: 3, 3: 2, 2: 1, 5: 1})
c[3]
>>> 2
c[4]
>>> 0
c[5] += 1
>>> Counter({1: 3, 3: 2, 5: 2, 2: 1})
c + Counter({1: 2, 2: 3})
>>> Counter({1: 5, 2: 4, 3: 2, 5: 2})
c - Counter({1: 2, 2: 3})
>>> Counter({3: 2, 5: 2, 1: 1})
c.most_common(2)
>>> [(1, 3), (3, 2)]
c.update({5: 1})
>>> Counter({1: 3, 3: 2, 5: 2, 2: 1})
c.subtract({2: 4, 3: 6})
>>> Counter({1: 3, 5: 2, 2: -3, 3: -4})
+c
>>> Counter({1: 3, 5: 2})
-c
>>> Counter({3: 4, 2: 3})
OrderedDict
'collections.OrderedDict' is a subclass of dict that preserves the insertion order of keys
Here are its main features and uses:
main features
- Guaranteed Ordering # Unlike a standard `dict` (since Python 3.7+, the order is also guaranteed for regular dictionaries, but `OrderedDict` offers additional methods for manipulating the order)
- Compatibility # Works like a regular dictionary, but with methods dedicated to ordering
- Performance # Slightly slower than a standard `dict` for basic operations, but useful for specific cases
specific methods
move_to_end(key, last=True) # Moves a key to the end (`last=True`) or the beginning (`last=False`) of the order
popitem(last=True) # Deletes and returns an element: the last (last=True) or the first (last=True=False)
defaultdict
'ollections.defaultdict' is a subclass of 'dict' that allows you to define a default value for non-existent keys. Here's how to use it and its main features:
Possible default values:
- int # returns 0
- list # returns an empty list []
- str # an empty string ""
- set # returns an empty set set()
- a custom function (e.g., lambda: "default_value")
Notes
Thread-safe: defaultdict is not thread-safe. For multithreaded use, use a dict with a lock or other synchronization.
Performance: Slightly slower than a classic dict for simple accesses, but much more practical for cases where keys are dynamic.
example
d = defaultdict(int)
d["apples"] += 1 # if "apples" doesn't exist, the default value is 0
d = defaultdict(list)
d["fruits"].append("apple")https://pypi.org/
https://pypi.org/project/autopep8/
indentation
- prefer space than tabulation
- use 4 spaces by indentation
- use : after commands if else elif def
x = y + z
def foo(x, y):
def foo(
x,
y
):
foo(x, y, z)
foo(
x,
y
)
x=[
: 0,
: 1,
]
pass
Do nothing but in a accepted syntax, void foo
def foo():
pass
while condition:
pass
if condition:
pass
else:
foo()
comments
# single line
""" single line """
"""
multiple
lines
"""
variables & attributes
authorized characters for name
a-z A-Z 0-9 _
authorized characters for the beginning of name
a-z A-Z
reserved words
and as assert break class continue def del elif else except false finally for from global if import in is lambda none nonlocal not or pass raise return true try while with yield
https://docs.python.org/3/library/stdtypes.html#string-methods
STRING
text.split(sep)
split string with a separator
'sep'.join(list)
join a list with a separator
text.replace(sch,str,count)
replace text by another with an optionnal counter replace
text.replace(sch,str).replace(sch2,str2)
text.strip()
strip all occurences of space,tab... in string
in text
return boolean, true if string are found in text
not in text
the opposite of in
text.find(sch)
return the first place of first occurence of a string in another (return -1 if not found)
text.rfind(sch)
return the first place of last occurence of a string in another (return -1 if not found)
index()
equal at find but launch an exception if not found
try:
"abcdefcdefghefghijk".index("zoo")
except Exception as e:
print("OOPS", type(e), e)
text.count(sch[, start[, end]])
count he number of ocuurences of text in string
text.startswith(sch)
return boolean, true if text are started with string
text.endswith(sch)
return boolean, true if text are ended with string
text.upper()
change all characters to uppercase
text.lower()
change all characters to lowercase
text.swapcase()
swat case for all characters
text.capitalize()
put first character in text in uppercase
text.title()
put first character of each 'words' in text in uppercase
https://docs.python.org/3/library/string.html#formatstrings
print()
print any types of elements
print(1, 'a', 12 + 4j, clas)
# replace the EOL by one space
print("une", "autre", end=' '); print('ligne')
F-PRINT
Evaluate {} and return text (don't print)
f"dans 10 ans {prenom} aura {age + 10} ans"
f"nous avons pour l'instant {len(notes)} notes"
f"pi avec seulement 2 chiffres apres la virgule {pi:.2f}"
f"{prenom:<10} -- {nom:^12} -- {solde:>8} €"
Left center right
< ^ >
FORMAT
Evaluate {} & print
"{} {} a {} ans".format(prenom, nom, age)
"{le_prenom} {le_nom} a {l_age} ans".format(le_nom=nom, le_prenom=prenom, l_age=age)
"pi avec seulement 2 chiffres apres la virgule {flottant:.2f}".format(flottant=pi)
Les données à afficher
comptes = [
('Apollin', 'Dupont', 127),
('Myrtille', 'Lamartine', 25432),
('Prune', 'Soc', 827465),
]
for prenom, nom, solde in comptes:
print(f"{prenom:<10} -- {nom:^12} -- {solde:>8} €")
%
"%s %s a %s ans" % (prenom, nom, age)
variables = {'le_nom': nom, 'le_prenom': prenom, 'l_age': age}
"%(le_nom)s, %(le_prenom)s, %(l_age)s ans" % variables
INPUT
Get answer always in a string !!
nom_ville = input("Entrez le nom de la ville : ")
print(f"nom_ville={nom_ville}")
int(input("Nombre de lignes ? ")) + 3STRING
s[start:end:step]
manipulate string by indexes, first included, second excluded (even with negative step)
s[start:] # from index to end
s[:end] # fro mstart to index
s[:] # duplicates element
s[start:-end] # from start to (end from last)
s[-start:-end] # from (start from last) to (end from last)
s[-start:end] # from (start from last) to end -> empty
s[-start:end:-step] # from (start from last) to (end from start) by rollback
s[::-1] # reverse
s[start;start] = [n] # insert element n in s at position start
s[start:end] = [] # unset elements between start and end (start included & end excluded)
LIST
Manipulate list with slice
l = [0, 2, 4, 8, 16, 32, 64, 128]
l
l[-1:1:-2]
l[2:4] = [100, 200, 300, 400, 500]
l
NUMPY
import numpy as np
un_cinq = np.array([1, 2, 3, 4, 5])
un_cinq
array = 10 * un_cinq[:, np.newaxis] + un_cinq
array
centre = array[1:4, 1:4]
centre
coins = array[::4, ::4]
coins
tete_en_bas = array[::-1,:]
tete_en_basA simulator: https://pythex.org
REGEXP - MATCH
import re
import re.MatchObject class
raw-string
raw-string: don't interprete '\' in regexp
re.match(regexp,text)
return the beginning matching of text
regexp = "(.*)-(.*)\.txt"
text = "abc-def.txt"
match = re.match(regexp, text)
print(match[1], match[2])
re.search(regexp,text)
return the matching of text
re.findall(regexp,text)
return all matching sequences
sentences = ['Lacus a donec, vitae gravida proin sociis.', 'Neque ipsum! rhoncus cras quam.']
for sentence in sentences:
print(f"---- dans >{sentence}<")
print(re.findall(r"\w*[am]\W", sentence))
re.split(regexp,text)
split string with a regexp
for sentence in sentences:
print(f"---- dans >{sentence}<")
print(re.split(r"\W+", sentence))
print()
re.sub(regexp,replace,text)
replace matching parts
for sentence in sentences:
print(f"---- dans >{sentence}<")
print(re.sub(r"(\w+)", r"X\1Y", sentence))
print()
re.compile(regexp).match(sample)
compile before matching
RESULT
search()
example: (?P<name>) affect name 'name' to matched text
# get
sample = " Isaac Newton, physicist"
match = re.search(r"(\w+) (?P<name>\w+)", sample)
# group return a list indexed (start at 1) by number & by name with (?P<name>)
# group() or group(0) return complete string
match.group()
match.groups() # list of groups
match.groupdict() # dict of all named groups
match.re()
The regular expression object whose match() or search() method produced this match instance
match.string()
The string passed to match() or search()
expand()
expand format in the string results of match
match.expand(r"last_name \g<name> first_name \1")
span()
span return index of matched text at the index of original text
begin, end = match.span('name')
sample[begin:end]
FLAG
IGNORECASE # ignore case
UNICODE # (alias U), use utf8 properties for \w sequence
LOCALE # (alias L), use current LOCAL for \w sequence
MULTILINE # (alias M), multiline
DOTALL # (alias S), catch all characters with . included EOL \n
regexp = "a*b+"
re_obj = re.compile(regexp, flags=re.IGNORECASE | re.DEBUG)
print(regexp, "->", nice(re_obj.match("AabB")))
SEQUENCE
[number]
Matches the contents of the group of the same number
alphanumeric
\w # Matches Unicode word characters
\W # Matches any character which is not a word character
\b # Matches the empty string, but only at the beginning or end of a word
\B # Matches the empty string, but only when it is not at the beginning or end of a word
\~ # Alphanumeric
\s # Matches whitespace characters including [ \t\n\r\f\v], and also many other characters
\S # Matches any character which is not a whitespace character
number
\d # Matches any decimal digit
\D # Matches any character which is not a decimal digit
others
\A # similar to ^, begining of text else in MULTILINE mode
\Z # similar to $, ending of text
regexp - greedy
by default * + ? match a longest chain found, ? allow to inverse the method to match the shortest string
*?
+?
??
GROUP
(...) # anonymous group
(?P<name>...) # named group
(?:...) # without group, optimize the render by not including content in entries table
(?P=name) # match text using predecessor matched text indexed with 'name'
(?=...)
(?!...)
(?<=...)
example: test the last named group id is equal at the first
regexp3 = "(?P<id>[0-9]+)[A-Za-z]+(?P<needle>[0-9]+)[A-Za-z]+(?P=id)"
examples
regexp_phone = r"(\+33|0)(?P<number>[0-9-]{9})\Z"
regexp_phone = r"(?i)\A(?P<proto>http|https|ftp|ssh)://((?P<user>[a-z]+)(:(?P<password>.+))?@)?(?P<hostname>[\w\._]+)(?P<port>\:\d+)?/(?P<path>[\w/]+)?\Z"https://docs.python.org/3/tutorial/datastructures.html#more-on-lists
INIT
l = [] # empty list
l= [e1, e2, e3, ..., en]
vec = [e] * n # n values
matrix = [[0]*n for i in range(n)] # n dimensions at n values
l = list(tuple) # cast
METHODS
l.append(element) # append only one elment at the end
l.extend([element1,element2]) # append elments of list at the end
l.insert(index,element) <=> l[index:index]=[element] # insert one element at index
l.remove(element) # remove the first occurence of the element
l.element=pop(index) # remove element at index & return element
l.reverse() # reverse elements
l.sort() # sort elements
l.sort(reverse=True) # sort items by reversing order
ls=sorted(l) # return a new sorted list of list passed in argument
[[l1.element1, l2.element1, ...], [l1.element2, l2.element2, ...] ] # associate lists
zip(l1, l2, ...) # return an iterator that aggregates elements from each of the iterables
COMPREHENSION
list2 = [treatment for i in list if cond]
receive string with numbers and other characters in multiple lines
have to be return a list of square separate by ':'
def carre(ligne):
import re
l = re.findall('(\d+)',ligne)
l = [int(i)**2 for i in l]
l = [str(i) for i in l]
return(':'.join(l))
received a list of numbers
return result of polynomial function
def P(x):
return(2*x**2 -3*x -2)
def liste_P(liste_x):
return([P(x) for x in liste_x])
graph
import numpy as np
import matplotlib.pyplot as plt
# un échantillon des X entre -10 et 10
X = np.linspace(-10, 10)
# et les Y correspondants
Y = liste_P(X)
# on n'a plus qu'à dessiner
plt.plot(X, Y)
plt.show()Shebang
# to launch a file directly from terminal console, without interpretor
#!/usr/bin/env python3
IDLE
install
sudo apt install idle && idle &
alt+p
for previous line in history
alt+n
for next line in history
Install
sudo apt install ipython3 && ipython3
Configuration
ipython3 profile create
file=~/.ipython/profile_default/ipython_config.py
[ -f "$file.keep" ] && cp -a "$file" "$file.keep$(date +%s)"
sch="c.InteractiveShell.color_info"; sed -i "/^#\?${sch}/ a ${sch} = True" "$file"
sch="c.InteractiveShell.colors"; sed -i "/^#\?${sch}/ a ${sch} = 'Linux'" "$file"
sch="c.TerminalInteractiveShell.highlighting_style"; sed -i "/^#\?${sch}/ a ${sch} = 'monokai'" "$file"
sch="c.TerminalInteractiveShell.highlight_matching_brackets"; sed -i "/^#\?${sch}/ a ${sch} = True" "$file"
!tail $file # access to bash commands with
Comments
# line
'''one line'''
'''
multiple
lines
'''
"""
multiple
lines
"""
print(value, ..., sep=' ', end='\n', file=sys.stdout, flush=False)
print(var)
print("a=",a," b=",b)
print(f"a={a}, b={b}")
Dir
dir(object)
return the list properties & methods of object
Type
isinstance(var,'type')
return instance type
ID
id(x)
return identifier of variable ('is' operator return True if id() is equal
Numeric
# example : 16 == 0x10 == 0o20 == 0b10000
0b # prefix binary
0o # prefix octal
0x # prefix hexadecimal
int # no size limit
float # 4.3, 15number 53b
complex # (float)+(float)j
bool # True 1 / False 0
Cast
int()
float()
complex()
bool()
bin()
Operators
=
+=
-=
/=
*=
**=
%=
<<=
>>=
+
-
*
**
/
//
%
Comparators
==
<
>
<=
>=
CLASS
fraction
from fractions import Fraction
Fraction(3,10) - Fraction(2,10) == Fraction(1,10)
decimal
from decimal import Decimal
Decimal('0.3') - Decimal('0.2') == Decimal('0.1')
Bitwise
operator
~
&
|
^
<<
>>
Mutable / Immutable
immutable
int
float
complex
bool
str
tuple
frozenset
bytes
mutable
list
bytearray
set
dict
class
...
Print code source
import inspect
lines = inspect.getsource(function)
print(lines)Suite de Fibonacci
result = []
num = 2
while (len(result) < 100):
i = 2
while (num % i) != 0 and i <= num: i += 1
if (i == num): result.append(num)
num += 1
print(result, len(result))