Python Basics

From AWF-Wiki
Revision as of 13:38, 20 July 2021 by Nnoelke (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

Python basics

What you see here is a jupyter notebook. It has text and code cells.

Shortcuts
run cell ctrl-enter
run cell and move to next shift-enter
activate editing enter
activate control mode esc
new cell above (in control mode) a
new cell below (in control mode) b
delete cell (in control mode) x
show documentation shift-tab

Jupyter is good for demo purposes - for coding I recommend pycharm.

This introduction is based on this tutorial

What is python?

  • python is a general purpose programming language.
  • It is an interpreted language, so there is a python process which reads your commands and translates them to machine code on the fly
  • It is an object oriented language (we'll see later what that means)

Example usages

  • web applications
  • deep lerning
  • statistics
  • image processing
  • writing scripts
  • robot control
  • ...

Python pros

  • readable
  • fast prototyping
  • platform-independent
  • packages for almost every application

Python limitations

  • Pure python is slow
  • you need libraries to make it fast
  • There are certain things you cannot program with it (e.g. operating systems)
  • As always, there are better languages for certain problems (e.g. for numerical simulations etc)

How to install

As we use jupyter notebooks for now this is kept brief:

  • Download anaconda
  • Use anaconda to create a so-called virtual environment
  • Activate the newly created environment
    • on linux: conda activate env_name
  • Install the packages you need into this environment
    • conda install packageA packageB packageC
  • if your package cannot be found in the anaconda cloud, use pip
    • pip install ...
  • Download pycharm
  • create a project folder
  • Tell pycharm to use the newly created environment for that project
    • settings -> interpreter settings -> navigate to .../anaconda/envs/your_env/bin/python3
  • start coding


1 Python syntax and functions

x = 1 + 1
print(x)
print("this is a string")
2
this is a string

python heavily relies on consistent indentation: indentation is used to define code blocks, always use the same number of spaces (mostly, just use tab)


# this is a comment
 
def foo(a, b):  # def signals that we define a function, note the colon at the end of the line
    return a+b  # note the indentation here, this is inside the function
 
print(foo(2,3))
 
 
# mostly, we use underscores to make long names readable
 
def nicely_documented_function(a: int, b: int) -> int:  # we can tell other programmers what types the function expects and which type it returns
    """This is a nicely documented function, so that we don't forget, what it does.
 
    This function adds two integer numbers. 
 
    Args:
        a (int): first integer
        b (int): second integer
 
    Returns:
        The sum of a and b.
    """
    return a+b
 
def function_with_default_arguments(a, b=1):
    return a+b
 
print(function_with_default_arguments(0))
5
1

A list of valid operators can be found here.

2 Variables

x = 1
y = x
y = y+1
print(x)
print(y)
 
a, b, c = 1, 2, 3
print(a,b,c)
d=e=1
print(d,e)
1
2
1 2 3
1 1

3 Most important data types

  • Numbers: integer, float, complex
  • List-like things: list, tuple, range
  • Bool (True/False)
  • Mappings: dict (dictionary)


3.1 Numbers

a = float(1)
print(a, " <- note the dot which indicates floating point numbers")
complex_number = 2+3j # j is the imaginary unit
print(complex_number)
1.0  <- note the dot which indicates floating point numbers
(2+3j)

3.2 Lists

Lists are collections of objects and can be changed after creation.


my_list = [9999, 2, 3, 4, 5]  # Use angled brackets for lists
print(my_list[0]) # accessing elements works also with angled brackets
print(my_list[2:-1])  # use colon to access ranges within the list
 
my_list[0] = 1  # change list elements
print(my_list[0])
 
my_list[0:3] = [0,0,0]
print(my_list)
 
other_list = [1, "cat", 3.14159]  # you can mix different types in one list, but I don't recommend that
print(other_list)
 
# how long is the list?
print("length:", len(my_list))
9999
[3, 4]
1
[0, 0, 0, 4, 5]
[1, 'cat', 3.14159]
length: 5
sum(my_list)
9

3.3 Tuples

Tuples are almost like lists, but they are read-only. Tuples are created by comma-separated values and parenthesis (although strictly speaking the parentesis can be left away).


t = (1,2,3)
print(t)
print(t[0:2])
t[0] = 999 # errors
(1, 2, 3)
(1, 2)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-541840c6b74c> in <module>
      2 print(t)
      3 print(t[0:2])
----> 4 t[0] = 999 # errors

TypeError: 'tuple' object does not support item assignment

3.4 Dictionary

A dict is a mapping from some value or object to another. It is created via curly brackets and colon-separated key-value pairs.


d = {"cat": "meow", "dog": "woof"}
d["cat"]
'meow'
d["dog"]
'woof'
# we can access all keys or values
d.keys()
dict_keys(['cat', 'dog'])
d.values()
dict_values(['meow', 'woof'])

4 Control flow

There are several methods for control flow in programming: for-loops, while-loops, if-else constructs etc


for i in range(4,8):
    print(i)
4
5
6
7

Note that ranges include the first element, but exclude the last.


i=0
while i <= 4:
    print(i)
    i += 1
0
1
2
3
4
for i in range(12,16):
    if i%2 == 0:
        print(i, "is even")
    elif i == 13:  # else if
        print(i, "is special because it is prime")
    else:
        print(i, "is odd")
12 is even
13 is special because it is prime
14 is even
15 is odd

You can combine comparison operators with and and not:


x = 1
y = 0
if x == 1 and y != 1:
    print("bla")
bla

You might have noticed that above we have used the in operator, which tells you whether the object on the left side is an element of the collection on the right side and can also be used to specify for loops.


None in [1,2,3, None]
True

5 Classes

Classes are a construct of object-oriented languages. They are used to store information along with valid functions for the respective class.

This might be a new concept for you so you might feel overwhelmed, which is ok :)


class Dog:
 
    # This is the init function which is used to initialize instances of a class, e.g. settinge names and parameters.
    def __init__(self, name):
        self.name = name  # the name of this very class instance (self) is 'name'
 
    # This is a function which is valid for all dogs
    def bark(self):  # this function behaves differently for each class instance, so we need 'self' as argument, which is automatically passed 
        print("{} makes WOOF".format(self.name))
d1 = Dog("Rex")
d2 = Dog("Fifi")
 
# call class methods, functions belonging to a class are called methods
d1.bark()  # here, d1 is automatically passed as 'self'
d2.bark()
Rex makes WOOF
Fifi makes WOOF
# access attributes via dot
d2.name
'Fifi'

5.1 Class inheritance

Dogs are animals, so maybe we can generalize some properties of dogs, which all other animals have too.

Class inheritance is used to group more general code in a base class and then add special functionality in derived classes.


class Animal:
    def __init__(self, name, num_legs, has_fur=False):
        self.name = name
        self.num_legs = num_legs
        self.has_fur = has_fur
 
    def make_noise(self):
        pass  # do nothing
 
 
class Dog(Animal):
    def __init__(self, name):
        super().__init__(name, 4, has_fur=True)  # here we initialize the attributes of the base class
 
    def __len__(self):
        return 42
 
    def make_noise(self):
        print("WOOF")
 
    def bite_mailman(self):
        print("grrr")
 
 
class Fish(Animal):
    def __init__(self, name):
        super().__init__(name, 0)
        self.has_flossen = True
 
    def make_noise(self):
        print("BLUB")
 
    def dive(self):
        #...
        pass
d1 = Dog("Rex")
f1 = Fish("Dori")
# now the same function does different things for different classes
d1.make_noise()
f1.make_noise()
WOOF
BLUB
len(d1)
42

6 Scopes

A scope defines where a certain variable is valid and from where it can be used.


# out here is the global scope
global_x = "global"
 
def foo():
    print(global_x)
 
foo()
 
def bar():
    local_x = "local"
 
    # we can define functions inside other functions which capture variables which live inside the outer function
    def inner():
        inner_x = "you can't call me from outside"
        print(local_x)
    inner()
    foo()
    #print(inner_x)  # this doesnt work
 
bar()
global
local
global

7 Modules

In python, functionality can be bundled into packages or modules. These modules can then be imported into your code or some other packages etc.

For example, the package numpy (numerical python) bundles most of python's math-related features and is implemented in Fortran. You will use it A LOT.


import numpy as np  # we can define a shorthand with 'as'

We can now access functions from numpy using the 'np' prefix. Keeping the prefix has two advantages: 1) it keeps the code tidy 2) you can directly see from which package a function comes


np.sin(np.pi/2)
1.0
from tensorflow import keras as K

8 Numpy Arrays

In remote sensing we deal with raster data a lot. These are usually represented as numpy arrays. Here's a brief overview:


# let's pretend we have a small 4 channel image with red, green, blue and near IR
arr = np.random.rand(100,100,4)  # 'random' is a submodule of numpy
print(arr)
[[[0.4783779  0.0666665  0.36632645 0.78756962]
  [0.84387339 0.08297951 0.56174942 0.07575504]
  [0.45929045 0.14816344 0.99976583 0.43759179]
  ...
  [0.33847892 0.59304289 0.33816897 0.85611047]
  [0.35929919 0.16102844 0.20939701 0.36630188]
  [0.48611541 0.89713767 0.9613248  0.3496362 ]]

 [[0.30897548 0.20392467 0.88429152 0.80235767]
  [0.7120191  0.98378301 0.92952877 0.64389345]
  [0.21180483 0.57641923 0.05639311 0.20657309]
  ...
  [0.41998292 0.35422083 0.30844408 0.36616702]
  [0.22144329 0.80292892 0.96134027 0.47812391]
  [0.72563568 0.11605323 0.329512   0.18940793]]

 [[0.60471739 0.42421692 0.68706924 0.24199528]
  [0.72184859 0.0715144  0.78216397 0.32048877]
  [0.77258736 0.27086924 0.44011665 0.72904129]
  ...
  [0.50012832 0.38660985 0.10314156 0.02345254]
  [0.30758998 0.94909693 0.92002871 0.73531702]
  [0.19297843 0.90905104 0.71683121 0.84214535]]

 ...

 [[0.72630635 0.52491659 0.11316555 0.87020299]
  [0.87439887 0.13777215 0.47056622 0.87441292]
  [0.5766554  0.80365538 0.86260712 0.97729407]
  ...
  [0.39479336 0.27902913 0.26474294 0.21777448]
  [0.77252769 0.84337445 0.77270321 0.04413112]
  [0.19972447 0.9096196  0.60168386 0.78122516]]

 [[0.2971643  0.02909061 0.47674666 0.38250887]
  [0.50246104 0.24160647 0.69544494 0.41333167]
  [0.60102428 0.94126009 0.18738681 0.22036723]
  ...
  [0.28213959 0.74568121 0.80216352 0.52367038]
  [0.68008696 0.36303433 0.29289413 0.09495144]
  [0.10411997 0.20476382 0.7842962  0.15009256]]

 [[0.14660409 0.91178269 0.16821274 0.07550217]
  [0.43466246 0.92489347 0.97904069 0.69493905]
  [0.79613184 0.81332517 0.3086502  0.27107484]
  ...
  [0.5652816  0.65105642 0.52476921 0.69492111]
  [0.78564702 0.12799144 0.24151924 0.20974478]
  [0.22885816 0.63371909 0.68368148 0.02969985]]]
type(arr)
numpy.ndarray
arr.shape
(100, 100, 4)
arr.astype("float32").dtype
dtype('float32')
arr.sum()
19941.407209301564
arr.mean()
0.4985351802325391
def ndvi(arr):
    return (arr[:, :, 0] - arr[:, :, 3]) / (arr[:, :, 0] + arr[:, :, 3] + 1E-10)  # you get all elements along one dimension via :
new_arr = ndvi(arr)
new_arr.shape
(100, 100)

Arrays behave a bit differntly than ordinary variables.


arr2 = arr  # here, arr2 is just another name for arr
arr2[:,:,0] = 0  # changing arr2 also changes arr
arr
array([[[0.        , 0.0666665 , 0.36632645, 0.78756962],
        [0.        , 0.08297951, 0.56174942, 0.07575504],
        [0.        , 0.14816344, 0.99976583, 0.43759179],
        ...,
        [0.        , 0.59304289, 0.33816897, 0.85611047],
        [0.        , 0.16102844, 0.20939701, 0.36630188],
        [0.        , 0.89713767, 0.9613248 , 0.3496362 ]],

       [[0.        , 0.20392467, 0.88429152, 0.80235767],
        [0.        , 0.98378301, 0.92952877, 0.64389345],
        [0.        , 0.57641923, 0.05639311, 0.20657309],
        ...,
        [0.        , 0.35422083, 0.30844408, 0.36616702],
        [0.        , 0.80292892, 0.96134027, 0.47812391],
        [0.        , 0.11605323, 0.329512  , 0.18940793]],

       [[0.        , 0.42421692, 0.68706924, 0.24199528],
        [0.        , 0.0715144 , 0.78216397, 0.32048877],
        [0.        , 0.27086924, 0.44011665, 0.72904129],
        ...,
        [0.        , 0.38660985, 0.10314156, 0.02345254],
        [0.        , 0.94909693, 0.92002871, 0.73531702],
        [0.        , 0.90905104, 0.71683121, 0.84214535]],

       ...,

       [[0.        , 0.52491659, 0.11316555, 0.87020299],
        [0.        , 0.13777215, 0.47056622, 0.87441292],
        [0.        , 0.80365538, 0.86260712, 0.97729407],
        ...,
        [0.        , 0.27902913, 0.26474294, 0.21777448],
        [0.        , 0.84337445, 0.77270321, 0.04413112],
        [0.        , 0.9096196 , 0.60168386, 0.78122516]],

       [[0.        , 0.02909061, 0.47674666, 0.38250887],
        [0.        , 0.24160647, 0.69544494, 0.41333167],
        [0.        , 0.94126009, 0.18738681, 0.22036723],
        ...,
        [0.        , 0.74568121, 0.80216352, 0.52367038],
        [0.        , 0.36303433, 0.29289413, 0.09495144],
        [0.        , 0.20476382, 0.7842962 , 0.15009256]],

       [[0.        , 0.91178269, 0.16821274, 0.07550217],
        [0.        , 0.92489347, 0.97904069, 0.69493905],
        [0.        , 0.81332517, 0.3086502 , 0.27107484],
        ...,
        [0.        , 0.65105642, 0.52476921, 0.69492111],
        [0.        , 0.12799144, 0.24151924, 0.20974478],
        [0.        , 0.63371909, 0.68368148, 0.02969985]]])

When you want to copy an array, you must do this explicitly:


arr3 = np.copy(arr)
arr += 1
(arr3 == arr).all()
False

9 Simple file IO

f = open("test.txt", "w") # open a new file in write mode 'w', other modes are 'r' for read and 'a' for append
f.write("Hello world!\n") # \n is the newline character
f.close() # always close files after opening
 
f = open("test.txt", 'r')
line = f.read()
f.close()
print(line)
Hello world!

There is another syntax which automatically closes files for you, which can make things safer and simpler.


with open("test.txt", 'a') as f:
    f.write("another line") # work with open file here
 
# here the file is no longer accessible
<_io.TextIOWrapper name='test.txt' mode='a' encoding='UTF-8'>
with open("test.txt", 'r') as f:
    print(f.read())
Hello world!
another lineanother line

In general the 'with ... as ...' syntax is there to automatically close things which should be closed, e.g. network connections


10 Error handling

Very often we make mistakes when coding. When running faulty code, the python interpreter throws errors, along with a stack trace which shows the origin of the error, so that we can fix it. However, there are also cases over which the programmer has no control, e.g. when trying to read a file, but the user has deleted it. To avoid a program crash, we can "catch" errors and tell the program what to do when errors happen.

A simple example:


1/0
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-82-9e1622b385b6> in <module>
----> 1 1/0

ZeroDivisionError: division by zero

Python throws a ZeroDivisionError


# simple function containing an error
def foo(x):
    return x + 1/0
 
def bar(x):
    return foo(x)
 
bar(1)
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-83-a5dd17cae51a> in <module>
      6     return foo(x)
      7 
----> 8 bar(1)

<ipython-input-83-a5dd17cae51a> in bar(x)
      4 
      5 def bar(x):
----> 6     return foo(x)
      7 
      8 bar(1)

<ipython-input-83-a5dd17cae51a> in foo(x)
      1 # simple function containing an error
      2 def foo(x):
----> 3     return x + 1/0
      4 
      5 def bar(x):

ZeroDivisionError: division by zero

The arrows on the left show where the error has happened with the most specific context last.

From top to bottom:

  1. The error happened in bar()
  2. The error happened in foo()
  3. The error happened on the line with 1/0

The very last line shows the error type.

Now we can tell the program to do something specific when an error is encountered. For this we use the try ... except ... syntax.


bar(1)
print("This code will never be reached.")
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-84-5fe9ae03030d> in <module>
----> 1 bar(1)
      2 print("This code will never be reached.")

<ipython-input-83-a5dd17cae51a> in bar(x)
      4 
      5 def bar(x):
----> 6     return foo(x)
      7 
      8 bar(1)

<ipython-input-83-a5dd17cae51a> in foo(x)
      1 # simple function containing an error
      2 def foo(x):
----> 3     return x + 1/0
      4 
      5 def bar(x):

ZeroDivisionError: division by zero
try:
    bar(1)   
except RuntimeError:
    pass
finally:
    print("Very important things that never error and must happen can be placed in a 'finally' block.\n")
 
print("Here we can continue happily")
Very important things that never error and must happen can be placed in a 'finally' block.

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-88-76e05b15239a> in <module>
      1 try:
----> 2     bar(1)
      3 except RuntimeError:
      4     pass
      5 finally:

<ipython-input-83-a5dd17cae51a> in bar(x)
      4 
      5 def bar(x):
----> 6     return foo(x)
      7 
      8 bar(1)

<ipython-input-83-a5dd17cae51a> in foo(x)
      1 # simple function containing an error
      2 def foo(x):
----> 3     return x + 1/0
      4 
      5 def bar(x):

ZeroDivisionError: division by zero

The except err block is executed, whenever the specific error err occurs in the try block. Important and safe cleanup operations go into the finally block.


We can also raise errors in case we want to avoid certain things:


x = 3
if x > 2:
    raise RuntimeError("x must not be larger than two, but it is {}".format(x))
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-89-a9c3e1ec0c76> in <module>
      1 x = 3
      2 if x > 2:
----> 3     raise RuntimeError("x must not be larger than two, but it is {}".format(x))

RuntimeError: x must not be larger than two, but it is 3

11 Generators

Python generators can be thought of as functions which remember their state. You have to initialize a generator before you can use it. They usually produce some form of data and you can call next() on them to get the next data. Instead of a return statement, generators have a yield statement. At this point the execution will be paused until the next request for data.


def test_gen(start, stop):
    i = start
    while i < stop:
        yield i # on the first call to next the execution runs up to here
        i += 1 # this will happen the first time at the second call to next
gen = test_gen(3,9) # initialization
next(gen) # call this multiple times to see the effect

Exercises

1) Write a function which calculates the volume of a sphere based on its radius.

assert volume(3) == 4/3*np.pi*27
def volume(r):
    return 4/3 * np.pi * r**3
 

2) Write a function which takes a list of strings and returns all strings starting with 'A'.

assert your_function(["Berta", "Alf", "Horst", "Adam"]) == ["Alf", "Adam"]
 

3) Write a function which takes two lists returns all elements that occur in both lists.

assert your_function([1,2,3,4,5],[3,4,5,6]) == [3,4,5]

4) Pro: Write a function which takes a number and performs a collatz iteration and counts the number of steps it takes to arrive at 1.

The Collatz problem is stated as follows:

  1. Start at an arbitrary integer number n larger than 0.
  2. If n is even, divide it by two.
  3. If n is odd, take 3n+1
  4. Repeat with the resulting number

For any number, this procedure will finally result in the sequence 4,2,1.


5) Write a function which calculates the NDVI of an RGB-NIR image (array) with dimensions channel-height-width

(nir - red) / (nir + red)


6) Write a class Box with attributes length, height and width and a method which returns the box volume.

Tips:

  • Remember that methods receive self as first argument.
  • You can access attributes of the 'object itself' via self. ... (dot)
  • The init method has two underscores on each side.

Pro: raise a RuntimeError if one of the dimensions is negative.


def collatz(n):
    steps = 0
    print(n)
    while n > 1:
        if n%2 == 0:
            n/=2
        else:
            n = 3*n+1
        steps += 1
        print(n)
    return steps
Personal tools
Namespaces

Variants
Actions
Navigation
Development
Toolbox
Print/export