Search
Integration Rules

Numerical Integration

Objectives

  • Learn to implement a few simple integration routines
  • Use built-in tools (Numpy and Scipy) to integrate
  • Study best practices

Standard imports

%matplotlib notebook
import numpy as np
import matplotlib.pyplot as plt

Aside: I wanted to setup a fun function to integrate. Here's how I played with a polynomial till I got one I liked. First, make a (mostly empty) plot to put our interactive output:

from ipywidgets import interact
x = np.linspace(0, 10)
fig, ax = plt.subplots()
(lines,) = plt.plot([], [])
ax.set_xlim(0, 10)
ax.set_ylim(-10, 10)
plt.show()
def f(a=(-1.0, 1.0), b=(-1.0, 1.0), c=(-10.0, 10.0), d=(-10.0, 10.0)):
    lines.set_data(x, np.poly1d([a, b, c, d, 0])(x))


interact(f)
<function __main__.f(a=(-1.0, 1.0), b=(-1.0, 1.0), c=(-10.0, 10.0), d=(-10.0, 10.0))>

Trapezoid

Okay, I have something I like. Let's compute:

$$ Ax^4 + Bx^3 + Cx^2 + Dx + E $$

Where $A=-0.1, B=1.0, C=-3.1, D=3.6, E=0.5$. I've just picked nice numbers to give an interesting plot and integral.

x = np.linspace(0, 5.5, 500)
f = np.poly1d([-0.1, 1.0, -3.1, 3.6, 0.5])
y = f(x)
x_i = np.linspace(0, 5.5, 4)
y_i = f(x_i)

fig, ax = plt.subplots()
plt.plot(x, y)
plt.plot(x_i, y_i, "o-r")
plt.plot(x_i, y_i * 0, "o-r")
plt.vlines(x_i, y_i * 0, y_i, "r")
plt.show()

Let's try integrating with several different values of N (picking odd values for later use - we could have even numbers if we wanted). Let's also compare with np.trapz.

for i in 3, 5, 7, 11, 21, 101, 1001:
    x_i = np.linspace(0, 5.5, i)
    y_i = f(x_i)
    h = np.diff(x_i)[0]
    my_intg = 1 / 2 * h * np.sum(y_i[:-1] + y_i[1:])
    of_intg = np.trapz(y_i, x_i)
    print(f"{i: 5} {my_intg: 16.12f} {of_intg: 16.12f}")
    3   8.197363281250   8.197363281250
    5  11.893682861328  11.893682861328
    7  12.707632056970  12.707632056970
   11  13.140031781250  13.140031781250
   21  13.325630892578  13.325630892578
  101  13.385421209428  13.385421209428
 1001  13.387891710433  13.387891710433

We can also calculate the exact value, since it's just a polynomial. In fact, since we built it with a specialized class in numpy, we can just call .integ to get the analytic integral (and then we call that with our points):

f.integ()(5.5)
13.387916666666657

Simpson's rule

We'll get the integrate subpackage from scipy - it has a lot of integration functions.

from scipy import integrate

Let's make sure we are selecting the times we want with our indexing expressions:

index = list(range(7))
index[:-1:2], index[1::2], index[2::2]
([0, 2, 4], [1, 3, 5], [2, 4, 6])

Looks good! Now let's try the rule, comparing with integrate.simps:

for i in 3, 5, 7, 11, 21, 101, 1001:
    x_i = np.linspace(0, 5.5, i)
    y_i = f(x_i)
    h = np.diff(x_i)[0]
    my_intg = 1 / 3 * h * np.sum(y_i[:-1:2] + 4 * y_i[1::2] + y_i[2::2])
    of_intg = integrate.simps(y_i, x_i)
    print(f"{i: 5} {my_intg: 16.12f} {of_intg: 16.12f}")
    3   9.193880208333   9.193880208333
    5  13.125789388021  13.125789388021
    7  13.336138438786  13.336138438786
   11  13.381206208333  13.381206208333
   21  13.387497263021  13.387497263021
  101  13.387915995621  13.387915995621
 1001  13.387916666600  13.387916666600

Gaussian quadrature

p = np.poly1d([7, -8, -3, 3])
x = np.linspace(-1, 1)
fig, ax = plt.subplots()
ax.plot(x, p(x))
plt.show()
p.integ()(1) - p.integ()(-1)
0.6666666666666674
p(-np.sqrt(1 / 3)) + p(np.sqrt(1 / 3))
0.6666666666666665

Optional: Use SymPy (Symbolic library for Python) to pretty print

from sympy.abc import x
import sympy

sympy.init_printing()
p(x).expand()
$\displaystyle 7 x^{3} - 8 x^{2} - 3 x + 3$
p(x).integrate(x).expand()
$\displaystyle \frac{7 x^{4}}{4} - \frac{8 x^{3}}{3} - \frac{3 x^{2}}{2} + 3 x$

Making Gaussian quadrature:

Slightly improved version:

max_in = 11

w = np.zeros(2001, dtype=np.float64)
x = np.zeros(2001, dtype=np.float64)


def func_e(x):
    return np.exp(-x)


def gauss(npts, job, a, b, x, w, vmin=0.0, vmax=0.0):
    m = i = j = t = t1 = pp = p1 = p2 = p3 = 0.0

    eps = 3e-14  # Accuracy: ******ADJUST THIS*******!
    m = (npts + 1) // 2
    for i in range(1, m + 1):
        t = np.cos(np.pi * (float(i) - 0.25) / (float(npts) + 0.5))

        t1 = 1
        while abs(t - t1) >= eps:
            p1 = 1.0
            p2 = 0.0
            for j in range(1, npts + 1):
                p3 = p2
                p2 = p1
                p1 = ((2.0 * float(j) - 1) * t * p2 - (float(j) - 1.0) * p3) / (
                    float(j)
                )
            pp = npts * (t * p1 - p2) / (t * t - 1.0)
            t1 = t
            t = t1 - p1 / pp

        x[i - 1] = -t
        x[npts - i] = t
        w[i - 1] = 2.0 / ((1.0 - t * t) * pp * pp)
        w[npts - i] = w[i - 1]

    if job == 0:
        for i in range(0, npts):
            x[i] = x[i] * (b - a) / 2.0 + (b + a) / 2.0
            w[i] = w[i] * (b - a) / 2.0

    if job == 1:
        for i in range(0, npts):
            xi = x[i]
            x[i] = a * b * (1.0 + xi) / (b + a - (b - a) * xi)
            w[i] = (
                w[i]
                * 2.0
                * a
                * b
                * b
                / ((b + a - (b - a) * xi) * (b + a - (b - a) * xi))
            )

    if job == 2:
        for i in range(0, npts):
            xi = x[i]
            x[i] = (b * xi + b + a + a) / (1.0 - xi)
            w[i] = w[i] * 2.0 * (a + b) / ((1.0 - xi) * (1.0 - xi))


def gaussint(no, min, max):
    quadra = 0.0
    gauss(no, 0, min, max, x, w)
    for n in range(0, no):
        quadra += func_e(x[n]) * w[n]
    return quadra


for i in range(3, max_in + 1, 2):
    result = gaussint(i, 0.0, 1.0)
    print(" i ", i, " err ", abs(result - 1 + 1 / np.exp(1)))
 i  3  err  3.0316449151079894e-07
 i  5  err  2.454703107446221e-13
 i  7  err  4.773959005888173e-15
 i  9  err  3.9968028886505635e-15
 i  11  err  1.0547118733938987e-14

Much better version:

def gauss_new(npts, job, a, b, vmin=0.0, vmax=0.0, eps=3e-14):

    x = np.zeros(npts, dtype=np.float64)
    w = np.zeros(npts, dtype=np.float64)

    for i in range((npts + 1) // 2):
        t = np.cos(np.pi * (i + 0.75) / (npts + 0.5))

        t1 = 1
        while abs(t - t1) >= eps:
            p1 = 1.0
            p2 = 0.0
            for j in range(npts):
                p3 = p2
                p2 = p1
                p1 = ((2 * j + 1) * t * p2 - j * p3) / (j + 1)

            pp = npts * (t * p1 - p2) / (t ** 2 - 1)
            t1 = t
            t = t1 - p1 / pp

        x[i] = -t
        x[~i] = t
        w[i] = 2 / ((1 - t ** 2) * pp ** 2)
        w[~i] = w[i]

    if job == 0:
        w = w * (b - a) / 2
        x = x * (b - a) / 2 + (b + a) / 2

    elif job == 1:
        # Use old x for w and x calcs
        w = w * 2 * a * b ** 2 / (b + a - (b - a) * x) ** 2
        x = a * b * (1 + x) / (b + a - (b - a) * x)

    elif job == 2:
        # Use old x for w and x calcs
        w = w * 2 * (a + b) / (1 - x) ** 2
        x = (b * x + b + 2 * a) / (1 - x)

    else:
        raise RuntimeError(f"Job ({job}) must be 0, 1, or 2")

    return x, w


def gaussint_new(f, no, min, max):
    quadra = 0.0
    x, w = gauss_new(no, 0, min, max)
    for n in range(no):
        quadra += f(x[n]) * w[n]
    return quadra


for i in range(3, max_in + 1, 2):
    result = gaussint_new(func_e, i, 0, 1)
    print("i", i, "err", abs(result - 1 + 1 / np.exp(1)))
i 3 err 3.0316449151079894e-07
i 5 err 2.454703107446221e-13
i 7 err 4.773959005888173e-15
i 9 err 4.107825191113079e-15
i 11 err 1.0547118733938987e-14

Using the classic QUADPACK from the Fortran days...

result, *_ = integrate.quad(func_e, 0, 1)
print(abs(result - 1 + 1 / np.exp(1)))
1.1102230246251565e-16

Using fixed_quad, which should be fairly similar to our algorithm:

for i in range(3, max_in + 1, 2):
    result, *_ = integrate.fixed_quad(func_e, 0, 1, n=i)
    print("i", i, "err", abs(result - 1 + 1 / np.exp(1)))
i 3 err 3.031644898454644e-07
i 5 err 2.404743071338089e-13
i 7 err 0.0
i 9 err 1.1102230246251565e-16
i 11 err 1.1102230246251565e-16

Look at the "See Also" section of the docstrings for the integration functions!