CTypes provide C compatible data types and allow function calls from DLLs or shared libraries without having to write custom C extensions for every operation.
CTypes are a Foreign Function Interface (FFI) library and provide an API for creating C-compatible datatypes.
To load a library, you can either instantiate one of the preceding classes with proper arguments or call the LoadLibrary() function from the submodule associated with a specific class:
Both library loading conventions (the LoadLibrary() function and specific library-type classes) require you to use the full library name. This means all the predefined library prefixes and suffixes need to be included. For example, to load the C standard library on Linux, you need to write the following:
>>> import ctypes
>>> ctypes.cdll.LoadLibrary('libc.so.6')
<CDLL 'libc.so.6', handle 7f0603e5f000 at 7f0603d4cbd0>
>>> import ctypes
>>> from ctypes.util import find_library
>>> ctypes.cdll.LoadLibrary(find_library('c'))
<CDLL '/usr/lib/libc.dylib', handle 7fff69b97c98 at 0x101b73ac8>
>>> ctypes.cdll.LoadLibrary(find_library('bz2'))
<CDLL '/usr/lib/libbz2.dylib', handle 10042d170 at 0x101b6ee80>
>>> ctypes.cdll.LoadLibrary(find_library('AGL'))
<CDLL '/System/Library/Frameworks/AGL.framework/AGL', handle 101811610 at 0x101b73a58>
#Linux
>>> import ctypes
>>> from ctypes.util import find_library
>>>
>>> libc = ctypes.cdll.LoadLibrary(find_library('c'))
>>>
>>> libc.printf(b"Hello world!\n")
Hello world!
Unfortunately, all the built-in Python types except integers, strings, and bytes are incompatible with C datatypes and thus must be wrapped in the corresponding classes provided by the ctypes module.
| ctypes type | C type | Python |
|---|---|---|
| c_bool | _Bool | bool (1) |
| c_char | char | 1-character bytes object |
| c_wchar | wchar_t | 1-character string |
| c_byte | char | int |
| c_ubyte | unsigned char | int |
| ctypes type | C type | Python |
|---|---|---|
| c_short | short | int |
| c_ushort | unsigned short | int |
| c_int | int | int |
| c_uint | unsigned int | int |
| c_long | long | int |
| c_ulong | unsigned long | int |
| c_longlong | __int64 or long long | int |
| c_ulonglong | unsigned __int64 or unsigned long long | int |
| c_size_t | size_t | int |
| ctypes type | C type | Python |
|---|---|---|
| c_ssize_t | ssize_t or Py_ssize_t | int |
| c_float | float | float |
| c_double | double | float |
| c_longdouble | long double | float |
| c_char_p | char * (NUL terminated) | bytes object or None |
| c_wchar_p | wchar_t * (NUL terminated) | string or None |
| c_void_p | void * | int or None |
As you can see, this table does not contain dedicated types that would reflect any of the Python collections as C arrays. The recommended way to create types for C arrays is to simply use the multiplication operator with the desired basic ctypes type:
>>> import ctypes
>>> IntArray5 = ctypes.c_int * 5
>>> c_int_array = IntArray5(1, 2, 3, 4, 5)
>>> FloatArray2 = ctypes.c_float * 2
>>> c_float_array = FloatArray2(0, 3.14)
>>> c_float_array[1]
3.140000104904175
If a C library requires arguments to be passed and/or has a return value, it is a good practice to declare them like so:
import ctypes
calc = ctypes.CDLL("MyCalc.dll")
addition = calc.addition()
# Declare return type
addition.restype = ctypes.c_int
# Declare argument types
addition.argtypes = [ctypes.c_int, ctypes.c_int]
# Pass arguments
addition(1,2)
Creating C Structs in Python can be useful. For instance, let's assume a function within a DLL requires a pointer to a struct to be passed. A real world example of this would be a DLL file that communicates with hardware. Below is an example of how to create a C struct using Python ctypes
import ctypes
# Create the struct
class P_Struct(ctypes.Structure):
_fields_ = [("field_1", ctypes.c_int),
("field_2", ctypes.c_char_p)]
# Pass struct values
my_struct = P_Struct(1, "Hello World")
# Create a pointer to my_struct
pointer_my_struct = ctypes.pointer(my_struct)
print my_struct.field_1, my_struct.field_2
# Import the DLL and pass pointer_my_struct to the C function requiring a pointer to a struct.
Output:
1 Hello World
Output:>
1 Hello World
import re
search_str = "This is a string to search"
re_search = re.compile("string")
matched_obj = re_search.search(search_str)
print matched_obj
<_sre.SRE_Match object at 0x02CBFA68>
import re
patterns = ['Enterprise', 'Death Star']
pharse = "The Enterprise is the flagship of the Federation."
for pattern in patterns:
print 'Looking for "{}" in "{}": '.format(pattern, pharse)
if re.search(pattern, pharse):
print "found a match!"
else:
print "no match!"
Looking for "Enterprise" in "The Enterprise is the flagship of the Federation.":
found a match!
Looking for "Death Star" in "The Enterprise is the flagship of the Federation.":
no match!
Python's re.match is similar to re.search. The main difference being that match searches only at the start of the string whereas search will apply the pattern to the entire string.
import re
text = "I turned myself into a pickle. I'm Pickle Riiiiick."
text2 = "I did not turn myself into a pickle. I am not Pickle Riiiiick."
match = re.match("I turned myself", text)
match2 = re.match("I did not", text2)
if match == None:
print 'Could not find "{}" in "{}"'.format(match.re.pattern, match.string)
else:
print 'Found "{}" in "{}"'.format(match.re.pattern, match.string)
if match2 == None:
print 'Cound not find "{}" in "{}"'.format(match2.re.pattern, match2.string)
else:
print 'Found "{}" in "{}"'.format(match2.re.pattern, match2.string)
Found "I turned myself" in "I turned myself into a pickle. I'm Pickle Riiiiick"
Found "I did not" in "I did not turn myself into a pickle. I am not Pickle Riiiiick."
Hashlib implements a common interface to many different secure hash and message digest algorithms.
import hashlib
m = hashlib.md5()
m.update("password")
print m.digest()
print m.hexdigest()
x = hashlib.md5()
x.update("password")
print m.digest()
if m.digest() == x.digest():
print 'access granted'
Struct performs conversions between Python values and C structs represented as Python strings. This is mostly used in handling binary data in files and network connections. This will be used during Network Programming.
import struct
data = struct.pack('hh1', 1, 2)
print data
data = struct.unpack('hh1', data)
print data
JSON is a lightweight data interchange format inspired by JavaScript object literal syntax. JSON is used in a whole lot of applications and scenarios.
import json
# JSON to Python
json_string = '{"make": "Ford", "model": "Mustang"}'
parsed_json = json.loads(json_string)
print parsed_json['make']
# Python to JSON
json_d = {
'make': 'Ford',
'model': 'Mustang',
'type': 'Pony',
'colors': ['red', 'blue', 'white', 'yellow']
}
parsed_d = json.dumps(json_d)
print parsed_d
Paramiko is a Python (2.7, 3.4+) implementation of SSHv2 protocol. Paramiko utilizes a Python C extension for low level crytography; though Paramiko itself utilizes a Python interface around SSH network concepts.
http://www.paramiko.org/pip is a tool for installing Python packages. There is all sorts of packages you can install from virtualenv (Python virtual environments), Requests (http library) to Scrapy (webscraping). Since you have Python already installed, you have pip. Below is an example on how to install virtualenv.
pip install virtualenv
Easy. Here is a great list of some of the most popular Python packages.
https://pythontips.com/2013/07/30/20-python-libraries-you-cant-live-without/Pickling is Python's way of serializing data. When you serialize data you're converting your object into a stream of bytes that you can save to a file for later use. pickle is part of the Python standard library. In order to access the module you must import it.
import pickle
You can pickle or serialize objects with the pick module, and you can also do the reverse which is known as unpickling, or deserializing the object.
After you've imported the pickle module the typical process is as follows:
You can pickle just about any type of object including:
In order to pickle an object we can use the following methods:
pickle.dump(object, file)
pickle.dumps(object)
The difference between dump() and dumps() is that the dump method will take an argument for a file you want to write your serialized object to, while the dumps method simply will return the pickled representation of the object as a bytes object. You can then later write it to a file if you so choose.
In order to unpickle our object we can use the following methods:
pickle.load(inputfile)
pickle.loads(data)
The difference between load() and loads() is that the load method will read the pickled representation of an object from the open file object, while the loads will return the representation of your data object.
NOTE: Remember that the pickle module is not secure. Only unpickle data that you trust. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling
The HitchHiker's guide is an excellent guide on Python that is constantly being updated.
http://docs.python-guide.org/en/latest/
Multithreading is similar to running multiple programs concurrently... but better.
A thread has a beginning, execution sequence and conclusion. An instruction pointer keeps track of where it is currently running.
import time
from threading import Thread
# Work to be done
def sleeper(i):
print "thread {:d} sleeps for 5 seconds\n".format(i)
time.sleep(5)
print "thread {:d} woke up\n".format(i)
# Creating the workers, and passing individual arguments to each of them
for i in range(10):
t = Thread(target=sleeper, args=(i,))
t.start()
thread 0 sleeps for 5 seconds
thread 1 sleeps for 5 seconds
thread 2 sleeps for 5 seconds
thread 3 sleeps for 5 seconds
thread 4 sleeps for 5 seconds
thread 5 sleeps for 5 seconds
thread 6 sleeps for 5 seconds
thread 7 sleeps for 5 seconds
thread 8 sleeps for 5 seconds
thread 9 sleeps for 5 seconds
thread 9 woke up
thread 7 woke up
thread 6 woke up
thread 4 woke up
thread 2 woke up
thread 1 woke up
thread 8 woke up
thread 0 woke up
thread 5 woke up
thread 3 woke up
Your output may not look like mine... that is okay. The time in which the workers start is almost always random due to how concurrent their launch sequences are.
Unit testing is a development process where parts of small code (called units) are individually and independently tested for proper operation. This can of course be done manually... but it is more often automated. This is an extremely important habit to build.
Python offers a powerful built in unit testing module called unittest.
import unittest
def addx(x):
return x + 1
class MyTest(unittest.TestCase):
def test(self):
# is addX(3) == 4?
self.assertEqual(addx(3), 4)
# Run test on file launch
if __name__ == '__main__':
unittest.main()
Output:
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK
In our case, this is exactly what we want to see. But what happens if the test case fails? Let's pass in the argument 4 to addx and see.
F
======================================================================
FAIL: test (__main__.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".\unit_testing.py", line 9, in test
self.assertEqual(addx(4), 4)
AssertionError: 5 != 4
----------------------------------------------------------------------
Ran 1 test in 0.001s
FAILED (failures=1)
The doctest module searches for sections of text that look like interactive Python sessions in docstrings, and then executes them to verify that they work.
def addition(x, y):
""" Return x plus y
>>> addition(3, 3)
6
>>> addition(5, 2)
1
"""
return x + y
if __name__ == '__main__':
import doctest
doctest.testmod()
**********************************************************************
File ".\unit_testing.py", line 6, in __main__.addition
Failed example:
addition(5, 2)
Expected:
1
Got:
7
**********************************************************************
1 items had failures:
1 of 2 in __main__.addition
***Test Failed*** 1 failures.
PS C:\Users\xconstaud\Documents\Python\re>
As you have guessed, this can be insanly useful. In most cases, spending the extra time unit testing will save you a lot more time over debugging.
Defined as “the class of a class”.
# in the most simple form
class Meta(type):
pass
# Python 2
class Final(object):
__metaclass__ = Meta
# Python 3
class Final(metaclass=Meta):
pass
__new__(): It’s a method which is called before __init__(). It creates the object and return it.
__init__(): This method just initialize the created object passed as parameter
Example:
class Meta(type):
def __init__(cls, name, bases, dct):
print "Creating class {} using Meta".format(name)
super(Meta, cls).__init__(name, bases, dct)
class Foo(object):
__metaclass__ = Meta
class FooBar(Foo):
pass
First we create a new metaclass called Meta:
Next, create a class called Foo… built from Meta rather than type.
Finally, create a FooBar class from Foo.